Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Motion Cues for Illustration of
Skeletal Motion Capture Data
Simon Bouvier-Zappa Victor Ostromoukhov
Pierre Poulin
LIGUM, Dept. I.R.O
Université de Montréal
Symposium on Non-Photorealistic Animation and Rendering, 2007
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
1 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Outline
1
Motivation
2
Motion Capture Hierarchy
3
Motion Cues
4
Results
5
Video
6
Conclusion
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
2 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Outline
1
Motivation
2
Motion Capture Hierarchy
3
Motion Cues
4
Results
5
Video
6
Conclusion
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
3 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Limitations Of The 2D Image
Conversion of 4D {x, y, z, t} information to 2D {x, y} data
Artists use visual cues to compensate inherent limitations of the
visualization medium.
Motion cues are visual cues emphasizing temporal information in
a static 2D representation.
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
4 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Limitations Of The 2D Image
Conversion of 4D {x, y, z, t} information to 2D {x, y} data
Artists use visual cues to compensate inherent limitations of the
visualization medium.
Motion cues are visual cues emphasizing temporal information in
a static 2D representation.
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
4 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Limitations Of The 2D Image
Conversion of 4D {x, y, z, t} information to 2D {x, y} data
Artists use visual cues to compensate inherent limitations of the
visualization medium.
Motion cues are visual cues emphasizing temporal information in
a static 2D representation.
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
4 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Limitations Of The 2D Image
Conversion of 4D {x, y, z, t} information to 2D {x, y} data
Artists use visual cues to compensate inherent limitations of the
visualization medium.
Motion cues are visual cues emphasizing temporal information in
a static 2D representation.
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
4 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Limitations Of The 2D Image
Conversion of 4D {x, y, z, t} information to 2D {x, y} data
Artists use visual cues to compensate inherent limitations of the
visualization medium.
Motion cues are visual cues emphasizing temporal information in
a static 2D representation.
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
4 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Visual Cues
1
Technical design
2
Cartoon illustration
3
Storyboard illustration
[Agrawala et al. 2003]
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
5 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Visual Cues
1
Technical design
2
Cartoon illustration
3
Storyboard illustration
[Collomosse et al. 2005]
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
5 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Visual Cues
1
Technical design
2
Cartoon illustration
3
Storyboard illustration
[Nienhaus et al. 2005]
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
5 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Visual Cues
1
Technical design
2
Cartoon illustration
3
Storyboard illustration
[Goldman et al. 2006]
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
5 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Criteria For The Motion Cue
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
6 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Criteria For The Motion Cue
Evocative
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
6 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Criteria For The Motion Cue
Evocative
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
Clear
NPAR 2007
6 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Criteria For The Motion Cue
Evocative
Clear
Robust
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
6 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Criteria For The Motion Cue
Evocative
Clear
Robust
Interactive
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
6 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Criteria For The Motion Cue
Evocative
Clear
Robust
Interactive
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
6 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Criteria For The Motion Cue
Evocative
Clear
Robust
Interactive
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
6 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
A Quick Example
Is this character in motion ?
If so, what is he doing ?
Motion cues can help
reconstitute past and future
motions.
Other cues will emphasize
subtle, otherwise
imperceptible, movements.
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
7 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
A Quick Example
Is this character in motion ?
If so, what is he doing ?
Motion cues can help
reconstitute past and future
motions.
Other cues will emphasize
subtle, otherwise
imperceptible, movements.
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
7 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
A Quick Example
Is this character in motion ?
If so, what is he doing ?
Motion cues can help
reconstitute past and future
motions.
Other cues will emphasize
subtle, otherwise
imperceptible, movements.
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
7 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Outline
1
Motivation
2
Motion Capture Hierarchy
3
Motion Cues
4
Results
5
Video
6
Conclusion
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
8 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Capturing Motion On A Single Bone
We extract a sequence of points at γ on the bone from pose t i − to
pose ti + to create a motion curve.
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
9 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Capturing Motion On A Single Bone
The motion curve gives satisfactory
results for translations and most rotations.
The motion curve cannot evaluate the
rolling value in a rotation.
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
10 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Possible Result Using Such Technique
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
11 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Subdivide The Skeleton In Joint Groupings
Form anatomically meaningful groups of bones
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
12 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Subdivide The Skeleton In Joint Groupings
Isolate articulated path
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
12 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Subdivide The Skeleton In Joint Groupings
Identify local root quaternion
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
12 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Subdivide The Skeleton In Joint Groupings
Define a point at distance γ on the articulated path
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
12 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Subdivide The Skeleton In Joint Groupings
Extract the “roll” value from the local root quaternion
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
12 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Modified Result With Improved Technique
Before
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
After
NPAR 2007
13 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Generalizing The Groupings Into A Motion Hierarchy
Recursively refine the groupings
No subdivision
1 subdivision, 2 groups
2 subdivisions, 5 groups
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
14 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Human Skeleton Hierarchy: Level 0
Body
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
15 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Human Skeleton Hierarchy: Level 1
Body
Upper Body
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
Lower Body
NPAR 2007
15 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Human Skeleton Hierarchy: Level 2
Upper Body
Torso
Head
Left Arm
Right Arm
Lower Body
Left Leg
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
Right Leg
NPAR 2007
15 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Human Skeleton Hierarchy: Level 3
Head
Torso
Head
Torso
Left Arm
Left Upper Arm
Left Forearm
Right Arm
Left Hand
Right Upper Arm
Left Leg
Left Upper Leg
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
Left Lower Leg
Right Forearm
Right Hand
Right Leg
Left Foot
Right Upper Leg
Right Lower Leg
NPAR 2007
Right Foot
15 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Outline
1
Motivation
2
Motion Capture Hierarchy
3
Motion Cues
4
Results
5
Video
6
Conclusion
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
16 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Motion Arrow: Analysis Of Motion Data
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
17 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Motion Arrow: Analysis Of Motion Data
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
17 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Motion Arrow: Analysis Of Motion Data
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
17 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Motion Arrow: Analysis Of Motion Data
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
17 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Motion Arrow: Analysis Of Motion Data
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
17 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Motion Arrow: Analysis Of Motion Data
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
17 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Motion Arrow: Analysis Of Motion Data
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
17 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Motion Arrow: Construction
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
18 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Motion Arrow: Construction
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
18 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Motion Arrow: Construction
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
18 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Motion Arrow: Parameters
v1
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
19 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Noise Waves: Analysis Of Motion Data
We calculate motion
amplitude using
bounding box of motion
curve.
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
20 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Noise Waves: Analysis Of Motion Data
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
21 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Noise Waves: Analysis Of Motion Data
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
21 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Noise Waves: Analysis Of Motion Data
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
21 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Noise Waves: Analysis Of Motion Data
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
21 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Noise Waves: Analysis Of Motion Data
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
21 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Noise Waves: Construction
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
22 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Noise Waves: Construction
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
22 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Noise Waves: Construction
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
22 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Noise Waves: Construction
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
22 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Noise Waves: Construction
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
22 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Noise Waves: Construction
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
22 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Stroboscopic Motion
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
23 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Outline
1
Motivation
2
Motion Capture Hierarchy
3
Motion Cues
4
Results
5
Video
6
Conclusion
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
24 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
A Soccer Player
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
25 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
A Weight-Lifter
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
26 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
A Weight-Lifter
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
26 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Multiple Key Poses
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
27 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Foot Steps
Technical illustration
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
Our result
NPAR 2007
28 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Outline
1
Motivation
2
Motion Capture Hierarchy
3
Motion Cues
4
Results
5
Video
6
Conclusion
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
29 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Video
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
30 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Outline
1
Motivation
2
Motion Capture Hierarchy
3
Motion Cues
4
Results
5
Video
6
Conclusion
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
31 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Summary
Combined use of motion arrows, noise waves and stroboscopic
motion to illustrate skeletal motion capture data.
Robust implementation of motion cues that adapt to the point of
view and the character’s geometry.
Interactive system to perform intuitive manipulations on motion
cues.
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
32 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Future Work
Short term
Integrate other motion cues to the system
Speed lines
Motion blur
Refine motion analysis tool for better pattern recognition
Perform user evaluation of our system
Long term
Automatically recognize specific movements from a database and
apply adapted motion cues
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
33 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Thank You !
Acknowledgements
Ubisoft
Philippe Beaudoin and Luc Leblanc for support and precious
insights !
Yann Rousseau for vector support !
All the people at the LIGUM !
Credits
Motion capture data: Carnegie Mellon University
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
34 / 35
Motivation
Motion Capture Hierarchy
Motion Cues
Results
Video
Conclusion
Thank You !
Bouvier-Zappa, Ostromoukhov, Poulin (Université de Montréal)
NPAR 2007
35 / 35