I. Pasciuto, A hybrid dynamic motion prediction method with collision detection
A hybrid dynamic motion prediction method with collision
detection
I. PASCIUTO*†, A. VALERO†, S. AUSEJO† and J.T. CELIGÜETA†
† CEIT and Tecnun (University of Navarra), San Sebastian, Spain
Abstract
In this paper we present a hybrid dynamic motion prediction method which combines data-based with
knowledge-based methods. On the one hand it relies on a database of captured motions for reference while on
the other it includes knowledge in the prediction as a dynamic motion control law. The motion prediction is
carried out imposing the conditions that the predicted motion must fulfill the goals defined for the motion;
resemble the reference motion in the database; follow a motion control law; and handle collisions with the
environment. Dynamics are included in the formulation both in the motion control law to be followed and in the
constraints the motion is subject to.
The method is applied to the prediction of an automobile clutch pedal depression and is validated against
repetitions of real motions performed by subjects of the same anthropometric characteristics as the subject used
in the prediction.
Keywords: Biomechanics, Posture and motion, Human motion prediction.
1. Introduction
Digital Human Models (DHMs) are more and more
frequently employed in product development
processes to test virtual environments in the early
stages of the design (Monnier et al. 2006; Monnier
et al. 2008; Abdel Malek and Arora 2008). To
simulate the interaction of various DHMs,
representing different user populations, with a
variety of environments, human motion prediction
is an interesting and useful tool.
Motion prediction aims at predicting within a
reasonable accuracy the motion that a subject would
perform to accomplish a specific goal. In contrast
with animation (Badler et al. 1993), which
essentially guides a particular subject with a
specific user-defined motion, motion prediction
focuses on representing the motions that a generic
specimen of a population would perform.
Due to the redundancy of degrees of freedom
(DoFs) in the human body, generally there are
infinite sets of values of the DoFs which fulfill a
specific task. Of all these sets of values, only some
can be defined as realistic, and may be grouped into
the various strategies and styles adopted by subjects
while carrying out the task. The challenge for
motion prediction is to simulate motions that not
only adjust to the requirements for realism but also
represent the variability observed in real motions.
*Corresponding author. Email: [email protected]
Motion prediction methods can be classified into
data-based
and
knowledge-based
methods,
according to whether they rely or not on a database
of captured motions.
Data-based methods (Zhang and Chaffin 2000;
Wang 2002; Monnier et al. 2003; Park et al. 2004;
Park 2008) consist in obtaining, among the motions
which compose the database, the most appropriate
one to be taken as reference and in modifying it in
order to fulfill the new motion constraints. The
motion in the new scenario (a new subject in a new
environment) is predicted starting from a real
motion carried out in a reference scenario (a
reference subject in a reference environment). The
main advantage of data-based methods lies in the
intrinsic realism of the reference motion, which
should be maintained during the modification
process. Furthermore, a database of captured
motions allows to identify the various strategies and
styles (Monnier et al. 2003; Park 2008) which the
subjects adopt during the fulfillment of the
considered task. Each identified strategy and style
may then be predicted, hence representing the
variety of ways in which the generic specimen of
the population may carry out the task. The main
drawback of data-based methods is the restriction of
being able to reasonably predict only tasks which
are present in the database. To predict a different
task, first a corresponding database of motions must
be generated.
1
I. Pasciuto, A hybrid dynamic motion prediction method with collision detection
On the other hand, knowledge-based methods (Kim
et al. 2006; Chung et al. 2007; Ren et al. 2007;
Abdel Malek and Arora 2008; Xiang et al. 2009) do
not rely on a database of captured motions and must
confer realism to the motion through the creation of
an appropriate objective function, representing the
motor control law that drives the motion. Although
knowledge-based methods are theoretically
applicable to any task to be performed, their
drawback lies in the difficulty of identifying the
correct objective function to describe the motion
control law. Even simple reaching motions seem to
require the combination of several objective
functions (Yang et al. 2004), leaving the prediction
of complex motions to be more than a challenge at
the present time.
For specific task-driven motions, as those involved
in the virtual testing of products, data-based motion
prediction methods may be considered the most
suitable option. Current data-based methods are
generally kinematic. However, for certain
applications, such as ergonomic studies, a dynamic
prediction may be required.
In this paper we present a hybrid dynamic motion
prediction method, which both relies on a database
of captured motions and includes knowledge in the
prediction as a dynamic motion control law. In
addition, collisions between the DHM and the
virtual environment are detected and modeled as
contact forces.
2. Method
The hybrid dynamic motion prediction method
proposed in this paper is optimization-based and is
composed of the following steps: human model
definition; database generation; reference motion
selection; end-effector trajectory modification;
resolution of the optimization problem describing
the conditions imposed to the predicted motion.
Each of these steps is hereafter described in detail.
2.1. Human model definition
To perform a human motion prediction, a DHM is
required. In this paper we consider a multibody
model of the human body, described with the
formalism of relative coordinates.
2.2. Database generation
Our prediction method relies on a structured
database of motions among which the reference
motion is identified. The database is generated
reconstructing captured motions; organizing them
according to the task, strategies, styles and scenario
characteristics; and analyzing them to identify the
relevant features of the motion.
An optoelectronic system is used to capture the
motions’ kinematics and the objects of the
environment with which the subjects interact are
equipped with force and pressure sensors to allow a
dynamic motion reconstruction through Inverse
Dynamics.
Once the motions are reconstructed, key-frames of
the motions are identified. A key-frame is a frame
in which a key event occurs in the motion. Keyframe identification is paramount in motion
prediction to define the goals that characterize the
key events in the motion and to determine at which
frames they must occur in the predicted motion. In
this paper we consider that key-frames in the
reference motion are maintained in the predicted
motion, i.e. key events occur at the same instant in
time.
The criteria for key-frame identification
encountered in literature are used for kinematic
motion prediction (Bindiganavale and Badler 1998;
Monnier et al. 2008), and only consider the
kinematic magnitudes of the motion, for instance
zero-velocity conditions for the end-effector. In
dynamic motion prediction, the key-frames
identified with kinematic criteria still represent key
events of the motion, but other key events may need
to be identified analyzing dynamic magnitudes.
Regarding dynamics-related key-frames, we
considered the force sensors’ values to identify the
frames at which the contact between the subject and
the environment occurs. The frames corresponding
to the first and last non-zero force values are the
key-frames corresponding to the beginning and the
end of the contact.
2.3. Reference motion selection
To predict the motion of a subject performing a
specific task in a given environment, the reference
motion must first be selected from the database.
The criterion adopted in this paper is based on the
comparison between the prediction scenario and the
scenarios of the motions that form the database for
the considered task. Among these motions, the one
that most resembles the prediction scenario is
retrieved.
The parameters that are used to carry out the
comparison are the subject’s gender, age and
anthropometrics; the position of the objects in the
environment with which the subject interacts as
well as their geometric and dynamic characteristics.
2.4. End-effector trajectory modification
Generally, the trajectory followed by the segment in
charge of fulfilling the task (usually an endeffector) in the reference motion does not lead to
the task’s accomplishment in the predicted scenario.
To guide the motion of the end-effector and ensure
the fulfillment of the task, a new trajectory for the
end-effector must be defined. A way of obtaining
an end-effector trajectory for the predicted motion
is to modify the end-effector’s reference trajectory
in order to meet the targets in the prediction
scenario.
2
I. Pasciuto, A hybrid dynamic motion prediction method with collision detection
Two modification methods are proposed in the
literature (Zhang 2002): one imposes that the
velocity along the predicted trajectory be
proportional to that of the reference motion (VP,
velocity proportional); the other imposes that the
acceleration along the predicted trajectory be the
same as in the reference motion (AP, acceleration
preserving). Generally both methods yield very
similar results, nonetheless VP is favored in this
study as it presents the desirable feature of
maintaining zero-velocity conditions at key-frames.
On the other hand, VP fails to generate a reasonable
trajectory when the initial and final positions of the
end-effector are very close to each other, since
numerical problems arise. In this case AP is applied
instead.
The end-effector trajectory modification must be
applied in each time period, which is the lapse of
time between two consecutive key-frames. Instead,
if the entire trajectory across a time period is
specified, we apply a different method. The method
imposes that the distance run by the end-effector
between two frames holds the same proportion
respect to the whole distance to be run in the time
period as in the reference motion.
2.5. Optimization
As mentioned in the introduction, the human body
is a highly redundant system, presenting more DoFs
than those strictly necessary to operate in the
workspace. For this reason, the imposition of the
end-effector trajectory is not enough to define the
motion of all the body segments, since infinite sets
of values of the DoFs allow its fulfillment.
Optimization may be used to define a criterion that
identifies the most realistic set of DoFs as the set
which minimizes a certain objective function
subject to a list of constraints.
As opposed to the optimization problems
encountered in data-based motion prediction
methods (Monnier 2003; Park et al. 2004) which
carry out an optimization for each frame in the
motion, we consider the motion as a whole and
search the optimum across the entire motion.
Instead of using the DoFs in each frame as design
variables, as in (Popović and Witkin 1999), we
chose to obtain a B-Spline representation of the
DoFs’ profiles. B-Spline curves are a linear
combination of linearly independent piecewise
polynomials, named basis functions, through
coefficients named control points. The basis
functions are a function of the independent variable
(in our case: time) and are non-zero only in limited
intervals in time. This leads to the property of local
support for B-Splines: the value of each control
point affects the curve in a lapse of time, which
generally is only a portion of the whole duration.
On the other hand, control points are defined in the
space of the dependent variables (in our case:
DoFs).
A B-Spline representation of a curve C(t) is given
by:
=
(1)
Ni p are the p-order basis functions, each associated
to a control point, CPi.
Our method consists in obtaining a B-Spline
representation of the ndof profiles of the DoFs in
the reference motion and in supposing that the same
number of control points is suitable to describe also
the DoFs’ profiles in the predicted motion. We
consider this supposition reasonable due to the
similarity criterion used to select the reference
motion from the database.
Given the set of ndof profiles which represent the
reference motion, we encounter the minimum
number nCP of control points needed to ensure that
each of the ndof B-Splines approximates the
reference DoFs’ profiles within a specified
tolerance (Piegl and Tiller 1997). The obtained set
of control points constitutes the initial
approximation for the optimization problem.
The advantage of a B-Spline parameterization of
the motion is that the whole motion is described by
a relatively small number of control points, which
are the design variables of the optimization problem
hereafter described.
The solution of optimization problem yields the
values of the control points which minimize an
objective function subject to constraints. The
objective function is in charge of maintaining a
resemblance to the reference motion and in defining
a motion control law to be followed. On the other
hand, the constraints we impose can be classified
into task-related constraints and balance constraints.
The former ensure that the goals in the motion are
fulfilled. The latter enforce the dynamic equilibrium
of the multibody model.
The objective function is defined as:
1
(2)
= Ψ Ψ
2
The vector Ψ is composed of equations representing
the following conditions:
• Resemble the DoFs’ profiles in the
reference motion:
(3)
−
∀
• Resemble the end-effector trajectory
obtained through the previously described
modification method:
(4)
−
• Follow the motion control law defined as
resembling the joint power profiles in the
reference motion:
(5)
∀
−
The motion control law described by Eq. (5) was
chosen among several motion control laws
encountered in literature (Yang et al. 2004; Kim et
al. 2006; Ren et al. 2007; Abdel-Malek and Arora
3
I. Pasciuto, A hybrid dynamic motion prediction method with collision detection
2008) since, according to (Pasciuto et al. 2010), it
seems to be the one which yields the most realistic
results.
The constraints the minimization is subject to are:
• Task-related: match the end-effector
trajectory in key-frames and when its
trajectory is defined across a time period:
(6)
−
= 0
• Balance: ensure the dynamic equilibrium
of the multibody model by imposing
balance conditions between the forces in
the root segment and the external forces
due to its interaction with the environment:
= 0 = 0 (7)
The dynamic magnitudes, used in the definition of
both the objective function (Eq. 5) and the
constraints (Eq. 7), are evaluated through the
application of the Newton-Euler recursive method
(Craig 2005). The method performs two iterations
across the system: the first starts from the root
segment and, moving outwards, it obtains the
segments kinematics; the second starts from the
distal segments and, moving inwards, it obtains the
forces and moments in the joints by imposing the
dynamic equilibrium of each segment.
To evaluate the external forces acting on the
segments, the collisions of the DHM with the
environment are detected and modeled. Collisions
are detected according to the position of the DHM’s
segments respect to the position of the objects with
which collisions are expected. If a collision takes
place, the collision is modeled as a force dependent
on the value of the displacement and on the velocity
of the segment. Both the displacement and the
segment velocity are a function of the control points
representing the DoFs’ profiles.
2.6. Test case
The method described in the previous section has
been applied to a clutch-pedal depression motion.
The human multibody model used to represent the
subjects performing the motion is that of a left leg,
modeled following RAMSIS specifications (Human
Solution GmbH). The model is composed of 4
segments (pelvis, thigh, shank, foot) and presents
13 DoFs: 3 translational and 10 rotational,
distributed across the joints as shown in Fig. 1.
The upper body of the DHM is taken into account
by conferring its inertial properties to the pelvis
segment.
Figure 1: RAMSIS model of a left leg with 3 translational
DoFs and 10 rotational DoFs.
A database of captured clutch-pedal operation
motions is available from the DHErgo project
(www.dhergo.org). The motions were carried out
by subjects belonging to 4 different populations:
young males and females (ages 20-35); elderly
males and females (ages 65-80). The subjects in
each population, approximately belonging to the
50th percentile, differ less than 10cm in height and
6kg/m2 in BMI.
Using our method we have predicted the motion of
a new subject in a modified environment. The new
scenario actually matches an experimental
configuration in which three motion repetitions
were captured. These repetitions are used for
validation. The characteristics of the new and
reference scenarios are detailed in Table 1.
Table 1: Characteristics of the reference and new
scenarios.
Subject
Vehicle*
Gender
Age
Height
Weight
Pedal rest pos. x
y
z
Travel length
Travel angle
Reference
Female
30
1.68m
59kg
-0.761m
-0.073m
-0.195m
0.161m
23º
New
Female
22
1.63m
63kg
-0.758m
-0.074m
-0.096m
0.158m
8º
*see Fig.2
The end-effector trajectory is imposed as a
constraint in the first frame and from the pedal
depression key-frame to the end depression keyframe. In all other frames it is included in the
objective function.
The force due to the clutch-pedal depression is
modeled as a function of the displacement (Fig. 3),
based on the pedal’s force vs. displacement curve,
which corresponds to a real clutch-pedal.
4
End Effector Trajectory X Axis [m]
I. Pasciuto, A hybrid dynamic motion prediction method with collision detection
-0.7
Reference
Modified
Predicted
-0.75
-0.8
-0.85
-0.9
0
20
40
60
80
100
Time [%]
Figure 2: Experimental car mock-up.
Figure 4: End effector trajectory along the x axis:
reference, modified and predicted trajectories.
End Effector Trajectory X Axis [m]
120
100
60
40
20
0.02
0.04
0.06
0.08
0.1
-0.9
0
20
40
Figure 3: Force-displacement curve for clutch-pedal.
The displacement depends on the position of the
foot, which depends on the DoFs of the model.
On the other hand, the force due to the interaction
with the seat is modeled as a function of the
penetration of the pelvis in the seat and its velocity.
Due to the local support property of B-Splines, both
the constraints and the optimization function are
evaluated only in specific frames in the motion.
Denoting with nFrames the number of frames in the
motion to be predicted and with nCP the number of
control points of the B-Spline representation, the
frames included in the optimization problem are a
sample every nFrames/nCP frames. Additionally,
all key-frames are included as well.
3. Results
The results obtained by applying the described
hybrid dynamic motion prediction method to the
test case are shown in the following figures.
All figures represent the values of the
corresponding magnitudes obtained with the motion
prediction (in red), the values in the reference
motion (in blue), and the values in the motions used
for validation (in black).
The time axis is normalized so that 50% of the
motion corresponds to the key-frame in which the
pedal depression begins and 100% corresponds to
the end depression key-frame.
60
80
100
Time [%]
0.12
Displacement (m)
Figure 5: End effector trajectory along the x axis:
predicted and validation trajectories.
90
GKNL Flex-Ext Angle [deg]
0
-0.85
Reference
Predicted
Validation1
Validation2
Validation3
80
70
60
50
40
0
20
40
60
80
100
Time [%]
Figure 6: Knee flexion-extension joint angle profile.
GHUL Flex-Ext Ang Vel [deg/s]
0
-0.8
60
Reference
Predicted
Validation1
Validation2
Validation3
40
20
0
-20
-40
0
20
40
60
80
100
Time [%]
Figure 7: Hip flexion-extension angular velocity profile.
150
GHUL Flex- Ext Torque [Nm]
Force (N)
80
Predicted
Validation1
Validation2
Validation3
-0.75
Reference
Predicted
Validation1
Validation2
Validation3
100
50
0
-50
-100
0
20
40
60
80
100
Time [%]
Figure 8: Hip flexion-extension torque profile.
5
I. Pasciuto, A hybrid dynamic motion prediction method with collision detection
maximum error across the motion is limited to less
than 10 N.
Force in Pelvis X Axis
150
100
50
Error
Child
Inertia
External
5. Conclusions
0
-50
-100
-150
0
20
40
60
80
100
Time [%]
Figure 9: Forces acting on the pelvis along the x axis.
4. Discussion
Fig.4 and Fig.5 show the trajectories followed by
the end-effector along the x axis. The curve in
green represents the modified trajectory which the
end-effector must resemble (in the first part of the
motion) or follow exactly (in the second part). This
trajectory was obtained with the described endeffector trajectory modification method. The
predicted motion (in red) matches the pedal
trajectory in the second half of the motion, as
imposed by the constraint. In the first half it must
resemble the modified trajectory while other
conditions must be met as well, hence the
resemblance is slighter.
Fig.5 compares the predicted trajectory to those
followed in the validation motions, and the values
obtained are quite similar. The differences in the
validation trajectories are partially due to the fact
that different points in the foot were used by the
validation subject to depress the pedal.
Fig.6 shows the knee flexion-extension joint angle
profile. The predicted motion resembles the shape
of the reference profile and the values obtained are
generally similar to those in the validation motions.
However in the predicted motion the knee seems to
flex more than in actually performed motions
before reaching the pedal (approximately at 30%).
Fig.7 and Fig.8 show the angular velocity and the
flexion-extension torque profiles at the hip joint. In
both, the predicted profiles seem to follow the
natural oscillations encountered in actually
performed motions and the predicted values are
always contained in the range of values of the real
motions.
The forces acting on the pelvis along the x axis are
shown in Fig.9. The oscillations in the first part of
the motion are due to the inertia forces of the
segment and of the child: these oscillations are
balanced by the seat. The second part of the motion
is characterized by the force acting on the hip joint,
mainly due to the pedal reaction force during
depression. The magenta curve shows the sum of
all the forces acting on the pelvis, i.e. the error of
Eq. (9). The frames at which the equilibrium
constraint was imposed present a strongly reduced
error ((o) 10-4 N), but at intermediate frames the
constraint is not met exactly. However the
In this paper we presented a hybrid dynamic motion
prediction method, which detects and models the
collisions between the DHM and the environment.
Dynamics are included both in the definition of the
objective function as well as in the constraints the
motion is subject to. This hybrid method relies on a
database of captured motions as well as on the
definition of a motion control law and is able to
predict the motion of a new subject performing a
task in a new environment.
The results we obtained do not differ substantially
from actually performed motions: on the one hand,
the ranges of values of predicted magnitudes are
similar to those encountered in real motions; on the
other, the oscillations in values that characterize
real motions are present in the predicted motion as
well.
Acknowledgement
The authors would like to thank INRETS for the
database of captured motions used in the present
study and Human Solutions GmbH for providing
the RAMSIS specifications to define the DHM.
This research was funded by the European project
DHErgo, “Digital Humans for Ergonomic design of
products” (FP7/2007-2013), grant agreement nº
218525.
References
Abdel-Malek K, Arora, J, 2008. Physics-based
digital human modeling: predictive dynamics. In:
Duffy,V.G. (Eds.), Handbook of Digital Human
Modeling. CRC Press- Taylor & Francis Group,
Boca Raton, Florida, USA, pp. 5.1-5.33.
Badler N.I, Phillips C.B, Webber B.L, 1993.
Simulating
Humans:
Computer
Graphics,
Animation and Control. Oxford University Press.
Bindiganavale R, Badler N.I, 1998. Motion
abstraction and mapping with spatial constraints.
Modelling and Motion Capture Techniques for
Virtual Environments. Springer, Berlin/Heidelberg,
pp. 70-82.
Chung H.J, Xiang Y, Mathai A, Rahmatalla S, Kim
J, Marler T, Beck S, Yang J, Arora J, Abdel-Malek
K, 2007. A robust formulation for prediction of
human running. In Proceedings of the 2007 Digital
Human Modeling for Design and Engineering
Symposium, Seattle, Washington.
Craig J.J, 2005. Introduction to Robotics:
Mechanics and Control. Pearson Prentice Hall.
Kim J.H, Abdel-Malek K, Yang J, Marler R.T,
2006. Prediction and analysis of human motion
dynamics performing various tasks. International
Journal of Human Factors Modelling and
Simulation Vol. 1, Nº 1, pp. 69-94.
6
I. Pasciuto, A hybrid dynamic motion prediction method with collision detection
Monnier G, Renard F, Chameroy A, Wang X,
Trasbot, J, 2006. A motion simulation approach
integrated into a design engineering process. In
Proceedings of the SAE International Conference
and Exposition of Digital Human Modelling for
Design and Engineering. July 4-6, Lyon, France.
Monnier G, Wang X, Trasbot J, 2008. A motion
simulation tool for automotive interior design. In:
Duffy,V.G. (Eds.), Handbook of Digital Human
Modeling. CRC Press-Taylor & Francis Group,
Boca Raton, Florida, USA, pp. 31.1-31.14.
Monnier G, Wang X, Verriest J.P, Goujon S, 2003.
Simulation of complex and specific task-orientated
movements - application to the automotive seat belt
reaching. In Proceedings of the SAE International
Conference and Exposition of Digital Human
Modeling for Design and Engineering. 17-19 June,
Montreal, Canada.
Park W, Chaffin D.B, Martin B.J, 2004. Toward
memory-based
human
motion
simulation:
development and validation of a motion
modification algorithm. IEEE Transaction on
System, Man and Cybernetics – Part A: Systems
and Humans Vol. 34, Nº 3, pp. 376-386.
Park W, 2008. Data-based human motion
simulation methods. In: Duffy,V.G. (Eds.),
Handbook of Digital Human Modeling. CRC PressTaylor & Francis Group, Boca Raton, Florida,
USA, pp. 9.1-9.17.
Pasciuto I, Valero A, Ausejo S, Celigüeta J.T, 2010.
A dynamic motion prediction method based on a
motion database and motion knowledge. In:
Proceedings of the 1st International Conference on
Applied Bionics & Biomechanics. Venice, Italy.
October 14-16 2010.
Piegl L.A, Tiller W, 1997. The NURBS Book.
Springer Verlag.
Popović Z, Witkin A, 1999. Physically based
motion transformation. In Proceedings of the ACM
SIGGRAPH 99, Computer Graphics Proceedings.
Ren L, Jones R.K, Howard D, 2007. Predictive
modelling of human walking over a complete gait
cycle. Journal of Biomechanics Vol. 40 Nº 7, pp.
1567-1574.
Wang X, 2002. Prediction of lower-limb
movements of clutch pedal operation from an
existing motion database. In Proceedings of 5th SAE
Digital Human Modeling Conference, Munich,
Germany, pp. 271-283.
Xiang Y, Chung H.J, Mathai A, Rahmatalla S, Kim
J, Marler T, Beck S, Yang J, Arora J.S, AbdelMalek K, 2009. Optimization-based dynamic
human walking prediction. International Journal for
Numerical Methods in Engineering, Vol. 79, Nº 6,
pp. 667-695.
Yang J, Marler R.T, Kim H.J, Arora J.S, AbdelMalek K, 2004. Multi-objective optimization for
upper body posture prediction. In Proceedings of
the 10th AIAA/ISSMO Multidisciplinary Analysis
and Optimization Conference. Albany, New York,
EEUU.
Zhang X, 2002. Deformation of angle profiles in
forward kinematics for nullifying end-point offset
while preserving movement properties. Journal of
Biomechanical Engineering Vol. 124, pp. 490-495.
Zhang X, Chaffin D, 2000. A three-dimensional
dynamic posture prediction model for simulating
in-vehicle
seated
reaching
movements:
development and validation. Ergonomics, Vol. 43,
Nº 9, pp. 1314-1330.
7