Wearable Kinesthetic System for Capturing and Classifying
Upper Limb Gesture
R. Bartalesi, F. Lorussi, M. Tesconi, A. Tognetti, G. Zupone and D. De Rossi
Interdepartmental Center “E. Piaggio”, University of Pisa, Italy
E-mail: [email protected]
Abstract
Electrically conductive elastomer composites (CEs)
show piezoresistive properties when a deformation is applied. In several applications, CEs can be integrated onto
fabric or other flexible substrate and can be employed as
strain sensors. Moreover, integrated CE sensors may be
used in biomechanical analysis to realize wearable kinesthetic interfaces able to detect posture and movement of the
human body. In the following a kinesthetic upper limb garment realized by CEs which allows to reconstruct shoulder,
elbow and wrist movements is presented.
1
Introduction
This work deals with the development of an innovative
measuring system devoted to the human movement analysis ([1]), with particular emphasis to remote rehabilitation
of stroke patients. In the present application, our efforts
have been focused in quantifying movements by using preprocessed output signals from CEs smeared on an elastic
fabric. Moreover a representation of the user’s movements
in an interactive tridimensional environment is given by using animation techniques based on quaternion algebra.
2
Kinesthetic Wearable Sensors and Kinematic Models of Human Joints
Figure 1:
The prototype for the upper limb.
the dynamical characterization, because the material shows
several non-linear peculiarities. Moreover, after a feedback
linearization, sensors need to be regulated to be used in our
applications. This matter is widely described in [2] and
some further results are exposed in the next section of the
present work. By using Lycra as a substrate we have obtained a sensing fabric which allows us to manufacture garments capable of monitoring human movements. In particular, by designing the spreading mask according to the
location of the joints we desire to monitor, we have obtained significative information from garments. Figure 1
shows the prototype realized for the upper limb, where all
sensors are represented by the segment series which compounds the bold track. Thin galley proofs constitute the
wiring system. Output signals from sensorized fabrics can
be used to drive a model of human kinematic chain. A
kinematic chain can be thought as a series of rigid segments connected by joints. In the present work we used
ideal joints in order to maintain a practical parameterization of movements (rotations) without trivializing the human joint movement. Moreover, all the kinematic models
developed in this work have been joined to form an Avatar
with human appearance (see figure 3).
The CEs we use are realized by a silicon rubber and
graphite mixture, smeared on an elastic fabric substrate according to the shape and the desired dimensions for the
sensors by using an adhesive mask. This technology provides both sensors and wiring by using the same elastic
material and avoids the use of obtrusive metallic wires
which may bound movements of the kinematic chain un3 Signal Processing and Software Analysis
der study. A technique devoted to compensate the piezoresistive properties of the CE wires is reported in [2] as well
A peculiarity in dynamical sensor response is the long
as the production process to obtain sensing substrate. The
transient time that occurs after a deformation. In order to
main properties of CE sensors are here summarized. The
reduce it, for coding human movements, a treatment of the
CE sensor gauge factor is about 2.8 and the temperature cooutput signals is essential. During the transient time which
efficient ratio is 0.08K −1 . Capacity effects showed by senfollows a step-in-length solicitation, output sensor signal
sors areofnegligible
up Eurohaptics
to 100M Hz.
Complexity
arises in on Haptic
can Interfaces
be approximated
a sum of and
decreasing
exponential
Proceedings
the First Joint
Conference
and Symposium
for Virtualas
Environment
Teleoperator
Systems
0-7695-2310-2/05 $20.00 © 2005 IEEE
Figure 3: Snapshot of KSS, the software developed to provide the sense
Figure 2:
The original signal is the output of three movements of the
stretching/unstreching of a sample. The modified signal shows the application of the algorithm for the long transient time reduction.
of movement in users wearing sensorized garments.
outputs, and the configuration space, Q, i.e. the lagrangian
coordinate space which describes the status of the kinematic model. We have implemented this map both by a
clusterization of the space S via a least square technique
into the space Q and by the interpolation of the discrete
map produced by the clusterization ([3]). In the present
application the first solution has been applied by using the
c 2
2
norm δi =
i=1 si − Sij as the clustering function,
where Sij ∈ S are the centers of the clusterization lattice
and si ∈ S represent the real value of the sensors. Moreover the software performs biomimetic animations, between clusterized positions by using a geometric representation in quaternion algebra: orientations acquired during
calibration in terms of Euler angles are translated in terms
of unit quaternions and transitions are defined through the
spherical linear interpolation algorithm (Slerp) described
in [4]. Using quaternions make animations fluids and realistic, unlike simple interpolating Euler angle or exponential map do. Moreover, the absence of singularities in unit
quaternions permits the execution of each arbitrary trajectory in the configuration space. In other words, each kind
of movement is representable. The system is under validation in stroke rehabilitation tasks. It is able to teach,
to evaluate and correct movements costituting a rehabilitation protocol. The objectivity of the data acquired on
patient performances consents to store and send them to
doctors and therapists which may evaluate tasks and patient progress, remotely.
functions
y(t) = c0 + c1 exp(−ω1 t) + ... + cp exp(−ωp t)
(1)
We have experimentally proved that the pole values ωi do
not depend on the deformation, but are a characteristic of
the sensors ([2]), so they can be determined once through
a calibration process. Coefficients ci s, are then calculated
by evaluating the functional
k
(yt −c0 +c1 exp(−ω1 t)+...+cp exp(−ωp t))2 (2)
t=1
where yt is the vector of acquired data and k is the number of samples considered in the regression. By minimizing eq. 2 with respect to ci , ωi we obtain the best regression in the least square sense. Since the pole values are
known, the only coefficients which have to be calculated
during a transient time are the ci (in particular c0 , which
represents the final value). To address this issue and eliminate noise deriving from the presence of high impedance
connecting wires ([2]), we have developed an algorithm
based on iterate integrations of equation 1. Coefficients
{ci }i=0...p are the solution of an over-dimensioned linear
system n × p, obtained by integrating n times eq. 1 on
the interval [t0 , tk ]. It is trivial to prove that the equation
system obtained is consistent for n ≥ p and k ≥ p. The
choice of n > p produces a filtering of signals while the
coefficients are calculated. A further stabilization is due
to the integration on all the interval where eq. 1 holds, by
collecting all the information previously stored. Figure 2
shows the application of this algorithm.
4
References
[1] De Rossi D., Lorussi F., Mazzoldi A., Orsini P., Sciligno E.P.:
Active Dressware: Wearable Kinesthetic Systems in Barth F.G.,
Humphrey J.A.C., Secomb T.W.: Sensor and sensing in biology
and engineering; Springer-Verlag Wien, 2003
Posture Recognition and Animation
[2] Tognetti A., Lorussi F., Tesconi M., De Rossi D. Strain Sensig Fabric Characterization; IEEE Sensor Conference, Wien,
2004
Kinematic Sensor System (KSS) is the software package which integrates both signal acquisition and process[3] Lorussi F. et al.: Wearable, redundant fabric-based sensor
ing described in section 3, providing a visualization of
arrays for reconstruction of body segment posture; IEEE Sensor
joint segment motion in a tridimensional interactive enviJournal, 2003
ronment (see figure 3). In order to perform posture recognition and representation KSS defines a map between the
[4] Shoemake K.: Animating rotations with quaternion curves;
sensor space,
S, Joint
which
contains Conference
the valuesand
of the
sensor on Haptic
ACM
SIGGRAPH
V.19,Is.3,pp
245-254,
Proceedings
of the First
Eurohaptics
Symposium
Interfaces
for Virtual
Environment
and 1985
Teleoperator Systems
0-7695-2310-2/05 $20.00 © 2005 IEEE
2