CS-184: Computer Graphics
Lecture #19: Motion Capture
Prof. James O’Brien
University of California, Berkeley
V2009-S-19-1.0
Today
• Motion
Capture
2
Tuesday, November 17, 2009
Motion Capture
• Record
• Use
motion from physical objects
motion to animate virtual objects
Simplified Pipeline:
Setup and calibrate
equipment
Record
performance
Process motion
data
Generate
animation
3
Basic Pipeline
Setup
Record
Process
Animatio
n
From Rose, et al., 1998
Tuesday, November 17, 2009
4
What types of objects?
• Human, whole
• Portions
• Facial
body
of body
animation
• Animals
• Puppets
• Other
objects
5
Capture Equipment
• Passive
Optical
•
Reflective markers
•
IR (typically) illumination
•
Special cameras
•
•
Fast, high res., filters
Triangulate for positions
Images from Motion Analysis
Tuesday, November 17, 2009
6
Capture
Equipment
Capture Equipment
Passive Optical Advantages
Accurate
use many markers
• Passive OpticalMay
Advantages
No
cables
• Accurate
High frequency
• May use many markers
• No cables
Disadvantages
• High frequency
Requires lots of processing
• Disadvantages Expensive (>$100K)
• Requires lots ofOcclusions
processing
• Expensive systems
Marker Swap
• Occlusions
Lighting/camera limitations
• Marker swap
• Lighting / camera limitations
7
Capture Equipment
• Active
Optical
Similar to passive but uses LEDs
• Blink IDs, no marker swap
• Number of markers trades off w/ frame rate
•
Phoenix Technology
Tuesday, November 17, 2009
Phase Space
8
Capture Equipment
• MagneticCapture
Trackers
Equipment
•
Transmitter emits field
•
Trackers sense
field Trackers
Magnetic
•
Trackers report
positionemits
and orientation
Transmitter
field
Trackers sense field
Trackers report location
and orientation
May be wireless
Control
9
Capture Equipment
• Electromagnetic Advantages
• 6 DOF data
• No occlusions
• Less post processing
• Cheaper than optical
• Disadvantages
• Cables
• Problems with metal objects
• Low(er) frequency
• Limited range
• Limited number of trackers
10
Tuesday, November 17, 2009
Capture Equipment
• Electromechanical
Analogus
11
Capture Equipment
• Puppets
Digital Image Design
Tuesday, November 17, 2009
12
Performance Capture
• Many
•
studios regard Motion Capture as evil
•
Synonymous with low quality motion
•
No directive / creative control
•
Cheap
Performance Capture is different
•
Use mocap device as an expressive input device
•
Similar to digital music and MIDI keyboards
13
Manipulating Motion Data
• Basic
tasks
Adjusting
• Blending
• Transitioning
• Retargeting
•
• Building
graphs
14
Tuesday, November 17, 2009
Adjusting
Nature of Motion Data
Why is this task not trivial?
Witkin and Popovic, 1995
Subset of motion curves from
captured walking motion.From Witkin and Popovic, SIGGRAPH 95
15
Adjusting
Adjusting
• IK
IK on single frames will not work
on single frames will not work
Gleicher,
SIGGRAPH
From
Gleicher,
SIGGRAPH 98
98
16
Tuesday, November 17, 2009
Adjusting
• Adjusting
Define desired
motion function in parts
Define desired function with
Adjustment
Inital sampled data
Result after adjustment
17
Adjusting
• Select
•
adjustment function from “some nice space”
Example C2 B-splines
• Spread
•
modification over reasonable period of time
User selects support radius
18
Tuesday, November 17, 2009
Adjusting
IK uses control
points of the Bspline now
Example:
position racket
fix right foot
fix left toes
balance
Witkin and Popovic SIGGRAPH 95
19
Adjusting
Witkin and Popovic SIGGRAPH 95
What if adjustment periods overlap?
20
Tuesday, November 17, 2009
Blending
Blending
• Given two
motions
make a can
motion
that combines
If given
two motions,
we blend
them
of bothto find a motion 1/2 between them?
qualities
• AssumeAssume
same DOFs
same DOFs
• AssumeAssume
same parameter
mappings
same parameter
mappings
21
Blending
• Consider
blending slow-walk and fast-walk
Bruderlin and Williams, SIGGRAPH 95
Tuesday, November 17, 2009
22
Blending
• Define
timewarp
Blending
functions to align features in motion
Define timewarp functions
Normalized time is w
23
Blending
Blending
Blending
in normalized time
• Blend inBlend
normalized
time
Blend in normalized time
Blend playback rate
• Blend playback
rate rate
Blend playback
24
Tuesday, November 17, 2009
Blending
• Blending
may still break features in original motions
Blending
Blending may still break "features" in
original motions
Touchdown for Run
Touchdown for Walk
Blend misses ground and floats
25
Blending
• Add
•
explicit constrains to key points
Blending
Enforce with IK over time
Add explicit constraints to key points
Touchdown for Run
Touchdown for Walk
26
Tuesday, November 17, 2009
Blending / Adjustment
• Short
edits will tend to look acceptable
• Longer
ones will often exhibit problems
• Optimize
to improve blends / adjustments
Add quality metric on adjustment
Minimize accelerations / torques
• Explicit smoothness constraints
• Other criteria...
•
•
27
Multivariate Blending
Blending
• Extend blending to multivariate interpolation
Extend to multivariate interpolation
"Speed"
“Speed”
“Happiness”
"Hippiness"
Weights are now barycentric coordiantes
28
Tuesday, November 17, 2009
Multivariate Blending
Blending
• Extend blending to multivariate interpolation
Extend to multivariate interpolation
"Speed"
If we have other examples
place them in the space also
“Speed”
“Happiness”
"Hippiness"
Becomes standard interpolation problem...
Use standard scattered-data
interpolation methods
29
Transitions
Transitioning
• Transition
from one motion
to another
Transition from motion A to motion B
Perform blend in overlap
region
30
Tuesday, November 17, 2009
Cyclification
• Special
• Both
case of transitioning
motions are the same
• Need
to modify beginning and end of a motion
simultaneously
31
Transition Graphs
Transition Graphs
Run
Flip
Sit
Walk
Stand
Trip
Dance
32
Tuesday, November 17, 2009
Motion Graphs
• Hand
build motion graphs often used in games
•
Significant amount of work required
•
Limited transitions by design
• Motion
graphs can also be built automatically
Transition Graphs
Run
Flip
Sit
Walk
Stand
Trip
Dance
33
Motion Graphs
• Similarity
•
metric
Measurement of how similar two frames of motion are
• Based on joint angles or point positions
• Must include some measure of velocity
• Ideally independent of capture setup and skeleton
• Capture
a “large” database of motions
34
Tuesday, November 17, 2009
Motion Graphs
• Compute
•
appearbetween
in the ACMall
SIGGRAPH
similarity To
metric
pairs of conference
frames proceedings
Maybe expensive
y objects in the
environ• Preprocessing
step
•
There may be too many good edges
sition and orientation at
Walking
Frame i
Frame j
Running
particular position (and
a specific time.
Walking , frame i
at satisfy the hard conves a graph search. Unmotions, the graph G has
d search of this graph is
ason is the need to enumany perfectly satisfacually well. For example,
end of a room at frame 0
s the room is very large,
onstraints.
Running, frame j
style (such as happy or
Clustering
35
Arikan and Forsyth, 2002
Figure 2: Every edge between two nodes representing different motion clips can be represented as a matrix where the entries correspond to edges. Typically, if there is one edge between two nodes
in our graph, there will be several, because if it is legal to cut from
one frame in the first sequence to another in the second, it will usually also be legal to cut between neighbors of these frames. This
means that, for each pair of nodes in the graph, there is a matrix
representing the weights of edges between the nodes. The i, j’th
entry in this matrix represents the weight for a cut from the i’th
h dynamic programming
the constraints
equally walksframe in the first sequence to the j’th frame in the second sequence.
• Random
The weight matrix for the whole graph is composed as a collection
s between equally valid
of blocks
this form.
Summarizing
graph involves compress• Start
ou dawdle at one
side inofsome part
of theofgraph
and randomly
makethe
transitions
ing
these
blocks
using
clustering.
. This suggests
summa• Avoid
dead ends
rser presentation that is
Useful
for “idling”
If this were not the case, our summary would rule out potential
G! . behaviors
hms are of no•help
here,
paths. In order to insure that this condition holds and because the
• Transitions
graph is very large, we cluster edges connecting every pair of nodes
cal times, we need to do
in G separately.
We cluster unconnected edge
groups of G from the
do the large •scale
moUse blending algorithm
we discussed
Domain of smoothing
P matrices (defined between every pair of nodes) using k-means
e details so that the moDiscontinuity Magnitude
joraxislength
ints as well as possible.
[Bishop 1995]. The number of clusters is chosen as ma
minoraxislength
while allowing us to exfor each group where the axis lengths refer to the ellipse that fits to
e path space. In such a
the cluster (obtained through Principal Component Analysis).
f the one finer level. Let
The nodes of G! are the same as the nodes of G. The edges con0.5
a hierarchical represennecting nodes in G! are cluster centers for clusters of edges connect0
s the finest. We will first
Signal
ing corresponding nodes in G. The centers Smoothed
are computed
by taking
!0.5
hierarchy to a path in G
Smoothing Function
the average of the edges in terms of f romFrame, toFrame and cost
values. At this point, every edge in G! represents many edges in G.
36
Figure
4: In the synthesized
motion,
discontinuities in orientation
We would like to have a tree of graph
representations
whose
root
are inevitable. We deal with these discontinuities using a form of
!
k-means
clustering
to
split
is G , and whose leaves are G. We use
localized smoothing. At the top left, a discontinuous orientation
Tuesday, November
2009
with its discontinuity
shown
at the top right. We now coneach17,
cluster
of edges in half at each signal,
intermediate
level and
obtain
struct
an interpolant
to this discontinuity,
shown on the bottom right
!
!!
!!!
n
can be represented in a
←to· ·the
·←
G ←
Gtofor
a hierarchical representation G ← Gand←addGit back
original
signal
get the continuous version
Motion Graphs
To appear in the ACM SIGGRAPH conference proceedings
Figure 5: Body constraints allow us to put “checkpoints” on the
motion: in the figure, the arrow on the right denotes the required
starting position and orientation and the arrow on the left is the required ending position and orientation. All constraints are also time
stamped forcing the body to be at the constraint at the time stamp.
For these two body constraints, we can generate many motions that
satisfy the constraints in real-time.
Motion graphs
• Match
•
•
•
•
•
•
imposed requirements
Start at a particular location
End at a particular location
Pass through particular pose
Can be solved using dynamic programing
Efficiency issues may require approximate solution
Notion of “goodness” of a solution
37
Suggested Reading
•
Fourier principles for emotion-based human figure animation, Unuma, Anjyo, and
Takeuchi, SIGGRAPH 95
•
Motion signal processing, Bruderlin and Williams, SIGGRAPH 95
•
Motion warping, Witkin and Popovic, SIGGRAPH 95
•
Efficient generation of motion transitions using spacetime constrains, Rose et al.,
SIGGRAPH 96
•
Retargeting motion to new characters, Gleicher, SIGGRAPH 98
•
Verbs and adverbs: Multidimensional motion interpolation, Rose, Cohen, and
Bodenheimer, IEEE: Computer Graphics and Applications, v. 18, no. 5, 1998
38
Tuesday, November 17, 2009
Suggested Reading
•
Retargeting motion to new characters, Gleicher, SIGGRAPH 98
•
Footskate Cleanup for Motion Capture Editing, Kovar, Schreiner, and Gleicher, SCA 2002.
•
Interactive Motion Generation from Examples, Arikan and Forsyth, SIGGRAPH 2002.
•
Motion Synthesis from Annotations, Arikan, Forsyth, and O'Brien, SIGGRAPH 2003.
•
Pushing People Around, Arikan, Forsyth, and O'Brien, unpublished.
•
Automatic Joint Parameter Estimation from Magnetic Motion Capture Data, O'Brien,
Bodenheimer, Brostow, and Hodgins, GI 2000.
•
Skeletal Parameter Estimation from Optical Motion Capture Data, Kirk, O'Brien, and
Forsyth, CVPR 2005.
•
Perception of Human Motion with Different Geometric Models, Hodgins, O'Brien, and
Tumblin, IEEE: TVCG 1998.
39
Tuesday, November 17, 2009