Real-Time Animation of a Virtual Tennis Player
Ching Ling Chin and Zhiyong Huang
Institute for Infocomm Research, A*STAR, Singapore
Email: {clchin, zyhuang}@i2r.a-star.edu.sg
Abstract—We describe our work in generating realistic animation for a virtual player in an interactive tennis game. In our
method, we use a compact database consisting of motion captured
locomotion and shot animations. A directed bipartite graph of
all possible locomotion-shot combinations is then formed. The
ball trajectory can be calculated after the opponent returns the
ball and hit possibilities can be found. Using the hit possibilities
as task requirements, we generate hypotheses of locomotionshot sequences, for which we formulate and solve for a decision
problem to find the best locomotion-shot sequence, taking the
naturalness and logic of the sequences in the context of a tennis
game into consideration. The best sequence is adapted using
root-level transformation, blending and speed modification. As
a result, we are able to retain the subtlety and nuances of the
motions by making minimal modifications to the animations in
the database while fulfilling the task of the virtual player. We
implement our method in an interactive tennis game and the
results confirm the effectiveness of our solution.
Index Terms - Motion Synthesis, Motion Capture, Animation with Constraints, Sports Simulation, Directed Graph,
Optimization
I. I NTRODUCTION
With the advance of computer graphics, the demand for
realism in games has become higher. Techniques that enhance
the user experience of the game with the aim of going real-time
are continuously being developed. In particular, to improve
realism in animation, data-driven animations are commonly
used. The use of data-driven animations is a conflicting choice
for games. On one hand, data-driven animations can create
realistic actions. On the other hand, it is a challenge to reuse
and modify the data to adapt to the different tasks that arise
as a result of interactivity in games. On top of this, how
the virtual character reacts to a situation is also important as
this contributes to its believability. In this paper, we propose
techniques to address realism, believability and interactivity
for a virtual tennis player.
A casual observation of a tennis game reveals that the
players’ techniques are very intricate. The top-notch players
have their own signature moves. We recognize that tennis
movements are so complex and deliberate that modifications
on the animation data may dilute the purpose of the stroke.
In order to generate the realism of the actions, we use motion
captured animations and choose to minimize modifications on
them to retain as much nuances as possible. Modifications
are done only at higher levels, such as speed change, simple
blending and root-level transformation. We then work around
the seemingly inflexibility to fulfill a task, i.e. to serve and to
respond to a returning ball.
Our animation database consists of pieces of unique actions, categorized into locomotion (running) and shot (hitting).
All combinations of these locomotion-shot sequences form a
directed bipartite graph. Based on the types of shots in the
database, we can also pre-define heights where the ball can be
hit. When a ball is returned from the opponent to the virtual
player, several hit possibilities are found by intercepting the
ball trajectory with the pre-defined heights. With the hit possibilities as input task requirements, hypotheses of locomotionshot sequences of various speed and transformation parameters
are generated. For each hypothesis, a set of pruning and
score functions eliminate and reward the candidate by taking
naturalness and logic into account. The best sequence, after
speed modification, root-level transformation and blending, is
then played on the virtual player, who will then fulfill a task
(hitting the ball) while looking natural and believable. This
method of finding the optimal sequences enables us to produce
realistic and believable animation for a real-time interactive
environment using a compact database.
In the next section, we will review some related works and
at the same time, explain our rationale for making minimal
modification on our data. Section III explains in greater detail
the animation control for a tennis game. We will conclude
with a brief description of our implementation and our goals
for the future.
II. BACKGROUND
The notion of creating sports games simulation is not new.
Noser and Thalmann [1] developed a tennis game simulation with synthetic actors or virtual players, controlling the
high level animation through a sequence of behaviours using
synthetic vision and auditory sensors. Their focus was in the
generation of the high-level behaviours rather than animation
control on an articulated figure.
One way to get the virtual player to hit the ball, which is
a goal-oriented task, requires some mechanism to modify the
original animation data. Some researchers modified the motion
capture data in various ways and degrees. One approach is
to place key frames in the original data and manipulate the
motion by warping the data while constraining to the keyed
information [2]. Zordan and Hodgins [3] combined motion
capture data with simulations by using control systems to
follow the motion capture data, while balancing the character
and modifying the animations to fulfil certain tasks. They
demonstrated their methods for boxing, fencing, dancing and
table tennis. In another research work [4], the character was
made to reach for an object and move the object to another
593
Proceedings of the Second APSIPA Annual Summit and Conference, pages 593–597,
Biopolis, Singapore, 14-17 December 2010.
10-0105930597©2010 APSIPA. All rights reserved.
position. Motion capture data information was incorporated
into the under-constrained inverse kinematics computation,
hence biasing the solution towards natural poses. The results
from these works are interesting. However, due to the nature
of simulation and inverse kinematics, computation times will
be of concern.
Then there are the physics-based solutions such as computing the key poses using special inverse kinematics to animate
the virtual opponent [5] or control-based simulations [6][7]
to perform certain tasks. Being generalized, this approach
has the advantage of being more editable than data-driven
approach. However, the biomechanics underlying the variation
of the tennis strokes [8][9] is complex. Reproducing tennis
strokes through purely physical-based methods or making
modifications to motion capture data may lose these important
nuances. Our approach is to keep the animations as intact as
possible, making only high level modifications such as speed
modification, blending and root level transformation.
The idea of using a directed graph for motions is well
established in Kovar et al [10]. We specialized the general
idea of directed graph to address the problem of fulfilling a
specific task in the tennis game context. Also similar to our
approach is Lee and Lee [11] in that they aim to find the best
solution from a motion capture database to perform a specific
task, such as hitting a target in a boxing game and playing
tennis. While they made use of the large motion database to
increase the realism of the avatar’s animation, we achieve the
variety of motions by categorizing short clips into their various
functions and then combining these clips to generate more
possibilities. As the researchers used a large motion capture
database, they approach the runtime problem by precomputing
the behaviours offline, and then used a control policy to decide
how the avatar should react in any situation. We do not use
any learning process. We also share some similar ideas with
Arikan et al. [12] who chose the desired motion from pieces
of motions by solving an objective function. We differ in the
formulation of our objective function which is customized for
tennis scenario.
III. A NIMATION
CONTROL OF THE VIRTUAL PLAYER
A. Model Set-up
As with a real-life tennis game, one would at least expect the
virtual player to be able to serve, run and hit a ball. We have
a customized skeletal set-up (Fig. 1) for our virtual player that
will enable it to perform goal-oriented tasks like serving and
hitting. The virtual player is made right-handed. The skeleton
contains 40 bones, its structure in most part common. What
is out of the ordinary are
• the extra bones placed along the length of the racket,
with one of them at the centre of the racket to indicate
the racket’s sweet spot.
• another extra bone created and extruded out of the left
hand’s palm. This will indicate the position of the ball
when the virtual player is holding a ball in its left hand.
This information is important for serving.
Fig. 1. Skeletal structure of the virtual player
B. Service
In a tennis game service, the tennis player usually bounces
the ball on the ground a few times, flings the ball into the
air and hit the ball with the racket. Reproducing this in the
virtual tennis game will require the animation of the virtual
player and the ball. The animation of the ball requires special
attention. It has to work in tandem with the animation of the
virtual player so that during playback, the ball leaves from and
bounces back into the left palm. The racket hits the flung ball
in mid-air at the right position and at the right time.
The path of the ball is subjected to this sequence: FOLLOWBOUNCE - FOLLOW - BOUNCE - ... - FOLLOW - FLING
- HIT. FOLLOW means that the ball is held by the left hand,
and the ball simply follows the position of the protruded left
bone. BOUNCE is the event when the ball leaves the hand
under an initial velocity, subjects under gravity, bounces off
the floor and back up to the left hand. FLING refers to the
event when the ball leaves the left hand under an upward initial
velocity, subjects under gravity, drops down after its velocity
turns zero and gets hit by the racket. After the HIT, the ball
will fly to the opposite court under a given initial velocity.
Fig. 2(a)(b) show the trajectories of the bouncing ball and the
ball in the air, respectively.
First, the times when the virtual player (a) bounces the ball
and receives the ball, (b) flings the ball into the air and then hit
it, have to be indicated. The durations of (a) or (b) can then be
found. By tracking the positions of the sweet spot bone and the
protruded left-hand bone at these times, the distance S, S1 , S2
can be found. Our objective here is to find the initial velocities
u1 and u2 of the BOUNCE and FLING events so that the path
of ball satisfies these durations and these distances. This can
be found by simple calculation of the quadratic trajectory and
the prior knowledge of gravity. Once the initial velocities are
known, we can generate the ball path at runtime, giving the
appearance of a ball in service. At the indicated time of impact,
when the virtual player swings the racket and hits the ball, the
ball is given another initial velocity that will throw it into the
next court. This hitting of the ball during service is similar to
the hitting of the ball during a shot which will be discussed
594
derive the angle θ and tL using a straightforward computation.
In order to find the best θ and tL , it is necessary to find the
transformed point R(θ)P~2,0 (tL ) of the path that will minimize
the error function e (θ, tL ) that is the squared distance from
R(θ)P~2,0 (tL ) to P~1 as follows:
e(θ, tL ) = kR(θ)P~2,0 (tL ) − P~1,0 k2
(a)
(b)
Fig. 2. (a) trajectory of a bouncing ball. (b) trajectory of a ball flung in the
air. The distances S1 , S2 , S can be found. The initial velocities u1 and u2
are to be calculated.
where P~1,0 = P~1 − P~0 , P~2,0 (tL ) = P~2 − P~0 , and P~2 is a point
along the path of the locomotion animation, which is also the
projection of the hip or root bone on the horizontal ground, at
time tL . The matrix R(θ) rotates the path at an angle θ with
point P~0 as the pivot.
The error function is dependent on two unknowns θ and tL ,
which we will solve iteratively. We set the initial conditions
with
kP~1,0 k
, where spL is the average speed of loco• t̂L =
spL
motion L. This initial estimate of t̂L is made under the
assumption that the locomotion L has constant velocity.
~2,0 (t̂L ), P~1,0 )
• θ = ∆θ̂ = ∠( P
~2,0 (tL )
• P̂2,0 (t̂L ) = R(∆θ̂)P
and solve iteratively
~1,0 )
• ∆θ̂ = ∠( P̂2,0 (t̂L ), P
′
• P̂2,0 (t̂L ) = R(∆θ̂)P̂2,0 (t̂L )
′
• search for a newly estimated t̂L such that
′
e = kP2,0
(t̂′L ) − P~1,0 k2
~0 at time t0 , the virtual player can hit the ball at
Fig. 3. Starting at position P
~S at time tS by applying the locomotion animation L rotated at an angle
P
~1 at time tL
θ and blending into a shot animation at P
in the next section.
C. Running and hitting
To explain our treatment of the animation of hitting an
incoming ball, we begin with a basic idea that is illustrated in
Fig. 3: suppose that the virtual player is positioned at P~0 at
time t0 and when the opponent hits the ball at t0 , we want to
apply a locomotion L, rotated at an angle θ from its original
motion direction, which is the moving direction in the raw
animation clip, so that the virtual player reaches P~1 at time
tL . Then a shot animation S is applied such that the virtual
player’s racket will hit the ball at position P~S at a fixed height
zS at time tS , or ∆tS after the change from the locomotion
to shot animation. ∆tS can be read from the raw animation
clip as the time when the player presumes to hit the ball.
When the opponent returns the ball at time t0 , the ball
trajectory is known. That is, the ball is known to reach P~b
at time tb . To ensure that the virtual player can hit the ball at
time tb , the speed of the locomotion-shot pair is modified so
that tS = tL + ∆tS = tb . Hence, only tL and θ need to be
computed.
Since the floor projection of the raw animation data is not a
straight line, nor is the motion at a constant speed, we cannot
(1)
(2)
is the minimum. In order to minimize the search computation, e is calculated in the vicinity of t̂L −τ < t′L < t̂L +τ ,
where 2τ is the size of the search window.
′
′
• let P̂2,0 (t̂L ) = P̂2,0 (t̂L ) and t̂L = t̂L and reiterate
The iteration stops when the error function falls below
a threshold or when the number of iteration loops exceeds
another threshold. We will then find tL as t̂L and θ as
∠(P~2,0 (t̂L ), P~1,0 ) This results in a closer distance between the
sweet spot of the racket and the ball at the point of impact.
D. Increasing the Realism
By itself, this method will not make the game very attractive
or natural as the virtual player will always use the same move
or run the same way. In a real racket game, there are several
kinds of moves (running sideway, forward, backward etc.) and
hitting strokes (forehand, backhand, smash etc.). A directed
bipartite graph of all combinations of the locomotion and shot
sequences can then be generated.
The variety of shot animations implies that there will be a
set of predefined heights where the ball can be hit and hence
for a given ball trajectory, there are a few shot possibilities.
Fig. 4 also shows that for a single shot, there are a few
possible strike points by finding the intersection of the the ball
trajectory with the height zS of the impact point of the shot
animation S. The hit possibilities are the task requirements
which determines the hypotheses that can be formed out of
the directed bipartite graph (Fig. 5). Each pair of locomotion
and shot sequence in each hypothesis has its own time tL
595
Fig. 4. Possible hit points at height zS from a single shot animation and a
given ball trajectory
Fig. 5. Forming hypotheses Hi from the directed bipartite graph of N
locomotions (L) and M shots (S) based on hit possibilities
and θ as mentioned in Section III.C., except that at this stage,
these values are estimated by assuming constant velocity in
the locomotion in order to reduce computation. Naturally, a
shot animation which does not have an intersection solution
with the ball trajectory will not be able to form a hypothesis
with its linked locomotions.
We now have a set of hypotheses {H0 , H1 , . . . , Hi }. What
is now necessary is a mechanism for formulating and solving
the decision problem of choosing an appropriate animation sequence. The decision problem is formulated as a combination
of pruning and score functions:

!
K−1
J−1
X
Y
Fj (Hi )
wk Gk (Hi )  . (3)
iopt = arg max 
i
j=0
k=0
The pruning functions Fj , where 0 ≤ j < J and J is the
total number of pruning function, are of binary nature in order
to prune away unwanted hypotheses. The second group is a
weighted sum of K score functions Gk where 0 ≤ k < K .
These are continuous functions which are set high to favour
desired traits. Weights wk can be included to this group of
functions to give more importance to certain function group.
In our implementation, there are three pruning functions:
F0 prunes hypotheses with out-of-court balls, while F1 and
F2 prune away unnatural locomotions. Unnatural locomotions
occur when the virtual player runs in a direction using an
animation meant for another direction (e.g. moving towards
the net using a backward running animation). These erroneous
directional choices will be pruned away by F1 . The second
type of unnatural locomotions are distance based. Some locomotion animations are meant to cover shorter distance and
some longer. Short distance animations will look odd if they
are cycled over a few times to cover a longer distance. These
locomotions will be pruned away by F2 .
Our implementation has two score functions, each represented by
|xk − uk |
Gk = 1 −
.
(4)
sk
The two sets of k, x, u, and s represent different parameters.
When the deviation from the optimal uk is greater than sk ,
then Gk becomes negative, penalizing unnatural animations.
These are:
• k = 0: we want to reward the smallest deviation from the
original motion capture speed.
• k = 1: we reward a particular shot given the distance to
where a shot is typically played. For example, a forehand
return is typically played at far court, whereas a drop shot
is usually played closer to the net.
After these steps of pruning and scoring, the surviving
hypothesis Hiopt with the highest score will be chosen for
animation playback after recalculating tL and θ, this time
using the iterative approach detailed in Section III.C. An
example of the result is shown in Fig. 6.
Fig. 6. A locomotion followed by a shot animation generated using our
method.
We have also added some features that will enhance the
realism of the game further. The virtual player can miss the
ball if the speed modification on the chosen animation is too
high. This reflects the real life situation that any tennis player
has a limitation on their running speed and hence will not be
able to reach a ball beyond their reach. The virtual player will
also look more energetic if it is made to run to its preferred
spot immediately after hitting the ball, similar to the strategies
of some tennis players.
IV. P RACTICAL I MPLEMENTATION AND R ESULTS
The animation module was implemented as part of the
immersive and interactive tennis game simulation described
in [13]. For demonstration and test purposes, the animation
module is also implemented for PC. The animation database
can vary in size, containing around 30 locomotions covering
varying distances, directions and speeds and around 10 shots
consisting of backhand and forehand strokes, motion-captured
from a professional tennis player.
596
In our actual tennis game simulation, a human uses a
tracked racket to play with the virtual player. The ball can
be hit and returned based on real-time tracking, collision
detection and rigid body physics. Thus, the ball is returned
in virtually infinite ways. In response, the virtual player runs
to the appropriate floor position and hits back the ball. The
interaction between the real and virtual players is like in any
real-life tennis game. Hundred of users have tried our tennis
game simulation and interacted with the virtual player. With
just a compact set of data, our method enables the virtual
player to handle infinite inputs based on interactions with the
real player.
V. C ONCLUSION
We have described our work on the animation of a virtual
player in a tennis game scenario with the aim to make the
animations real-time, realistic and believable. Motivated by
the fact that tennis movements are complex and the notion that
these nuances that are very difficult to reproduce, we chose to
keep animation data as intact as possible, making high level
edits where necessary. The animation control mechanisms for
serve and run-and-hit were described. We also explained the
generation of locomotion-shot sequence hypotheses from the
hit possibilities and a directed bipartite graph formed out of the
animations in the database. We then presented the formulation
and the solution of our decision problem to find the best run
and hit sequences that would make the virtual player look
natural and logical.
Having said that, our work still has room for improvement
and expansion. Currently, some manual preparation on the
animations is needed prior to loading them into the database.
An automation or even semi-automation of marking event
times would be ideal. Modification of the speed could also
vary over time for a smoother transition. Animation blending
could be improved and graph-based transition methods [14]
are worth looking at. Currently, we have a compact database
of animations, which contributes to the real-time performance
of the animation. We intend to expand the database with
even more animations to see how far the computation can
be pushed. We hope to include in our decision problem more
sophisticated requirements such as AI information and more
complex games rules. It will also be interesting to correlate
the ball trajectory with the dynamics of the shot chosen.
[2] A. Witkin, and Z. Popovic, “Motion warping”, Proceedings of ACM
SIGGRAPH 95, 105-108, 1995.
[3] V. Zordan, and J. Hodgins, “Motion capture-driven simulations that
hit and react,” Proceedings of the 2002 ACM SIGGRAPH/Eurographics
symposium on Computer animation, pp89-96, 2002
[4] K. Yamane, J.J. Kuffner, and J.K. Hodgins, “Synthesizing Animations
of Human Manipulation Tasks,” ACM Transactions on Graphics (SIGGRAPH 2004), 23(3), August 2004
[5] S. Rusdorf, G. Brunnett, M. Lorenz, and T. Winkler, “Real-Time Interaction with a Humanoid Avatar in an Immersive Table Tennis Simulation”,
IEEE Trans. On Visualization and Computer Graphics, 13(1): 15-25,
2007.
[6] S. Coros, P. Beaudoin, K. Yin, and M. van de Panne, “Synthesis of
Constrained Walking Skills,” ACM Transactions on Graphics, 27(5), Dec
2008.
[7] V. De Sapio, O. Khatib, and S. Delp, “Task-Level Approaches for the
Control of Constrained Multibody Systems,” Multibody System Dynamics,
vol. 16, no. 1, pp73-102, August 2006.
[8] D. Knudson, Biomechnical Principles of Tennis Technique: Using Science
to Improve Your Strokes, Racquet Tech Publishing, 2006
[9] H. Brody, R. Cross, and C. Lindsey, The Physics and Technology of
Tennis. Racquet Tech Publishing, 2004
[10] L. Kovar, M. Gleicher, F. Pighin, “Motion Graphs”, ACM Transactions
on Graphics (ACM SIGGRAPH 2002), Vol: 21, No: 3, pp 473–482, 2002
[11] J. Lee, and K.H. Lee, “Precomputing avatar behavior from human
motion data”, Graphical Models, v.68 n.2, 158-174, 2006
[12] O. Arikan, D.A. Forsyth, J. O’Brien. “Motion Synthesis from Annotations,” ACM Transactions on Graphics (ACM SIGGRAPH 2003), Vol: 22,
No: 3, pp 402–408, 2003.
[13] S.H. Xu, P. Song, G.G. Chua, C.L. Chin, Z.Y. Huang, and S. Rahardja,
“Tennis Space: an Interactive and Immersive Environment for Tennis Simulation”, 2009 Fifth International Conference for Image and Graphics,
icig, pp.652-657, 2009
[14] M. Gleicher, H.J. Shin, L. Kovar, A. Jepsen, “Snap-together motion:
assembling run-time animations,” I3D ’03: Proceedings of the 2003
symposium on Interactive 3D graphics, 181-188, 2003.
VI. ACKNOWLEDGEMENTS
This work was funded under the Personal 3D Entertainment Systems Programme, Institute for Infocomm Research,
A*STAR, Singapore. The authors wish to thank members of
the Interactive Sports Game Engine Team in the Programme
for their help in providing the 3D models and setting up the
visualization and development platforms.
R EFERENCES
[1] H. Noser, and D. Thalmann, “Sensor Based Synthetic Actors in a Tennis
Game Simulation”, Proceedings of the 1997 Conference on Computer
Graphics International, p.189, 1997.
597