Computer Science 6752
Computer Graphics
Project Report:
“Articulated Human Hand Model with
Inter-Joint Dependency Constraints”
For: Dr. Edward Brown
By: Andrew Vardy
Due: 16 December 1998
Table of Contents
TABLE OF CONTENTS.............................................................................................................................. 1
INTRODUCTION.........................................................................................................................................2
APPLICATION ............................................................................................................................................... 2
CONTRIBUTION TO RESEARCH ..................................................................................................................... 3
PROJECT IMPLEMENTATION ......................................................................................................................... 3
MODEL DESCRIPTION AND BACKGROUND ..................................................................................... 4
SKELETAL ANATOMY OF THE HUMAN HAND .............................................................................................. 4
DEVELOPING MODEL COORDINATE FRAMES USING DENAVIT-HARTENBURG .............................................6
JOINT DEPENDENCY CONSTRAINTS .............................................................................................................8
Static Constraints ...................................................................................................................................8
Intra-Finger Joint Dependency Constraints........................................................................................... 8
Inter-Finger Joint Dependency Constraints ........................................................................................... 9
RESULTS AND CONCLUSIONS.............................................................................................................11
RESULTS .................................................................................................................................................... 11
CONCLUSIONS ...........................................................................................................................................11
REFERENCES............................................................................................................................................13
1
Introduction
This report summarizes the background research, work activities, results, and conclusions that
came out of my term project in Computer Science 6752. The title of the project is, “Articulated
Human Hand Model with Inter-Joint Dependency Constraints”. The primary goal of this project
was to produce a model of the human hand that would allow manipulation of model parameters in
a manner consistent with the mechanics of the actual human hand. These parameters are
expressed as joint angles. The foreseen application for this model is a generate-and-test gesture
recognition system, which will be described shortly.
The mechanics of the joints in the human hand are to be modeled. The relevant information from
the physiology of the hand is that which answers the questionwhere are the axes of rotation that
create movement? The answer to this question will be given later, but let us now say that these
axes are located at the “movable joints” of the hand. Mathematically, the model is comprised of a
set of coordinate frames and the transformations that relate them. The model parameters are
components of these transformations. The coordinate frames were assigned to the movable joints
according to the Denavit-Hartenburg convention [1] employed in robotics.
This model also includes a number of constraints on model parameters. Some of these constraints
are static and simply describe ranges of allowable motion. Others are dynamic and introduce
dependencies between joints. This later category may be further subdivided into constraints that
act upon joints within the same finger, the intra-finger constraints, and those that act upon joints
within different fingers, the inter-finger constraints.
Application
The application originally envisioned for this model was a generate-and-test gesture recognition
system. This system is divided into two stages. In the first stage, images of an actual human
hand would be inputted. The system would take its internal hand model and manipulate its
parameters so that the model and the actual hand shared the same posture. This process will be
described in more detail shortly. The parameters of the matched model would be the output of
the first stage of the system. At the second stage, further processing could be done to classify a
particular posture, or sequence of postures, as a defined gesture. Classifying this gesture would
be the final output of the system. Thus, the model described in this paper would be used
primarily in the first stage of this system, but its parameters could also be used as a feature vector
for the classification stage of the system.
In the first stage of this system a two-dimensional projection of the model would be compared
with the actual hand image. If the resulting two images were judged to be too dissimilar then the
model parameters would be adjusted and a new projection taken. This process would continue
until the disparity between the projection and the image reached an acceptably low level. At this
point the model parameters would be sent on to the second stage of the system. Note, that this
second stage is a statistical pattern recognition system that is unrelated to computer graphics and
will therefore not be further discussed.
Thus, the parameters of the model will represent an n-dimensional space (n = number of
parameters) that needs to be searched. This is the motivation for the use and development of the
constraints described later. These constraints effectively prune the search space and improve the
system’s ability to find the correct hand posture in real time.
2
Contribution to Research
The only portion of this project which could be considered a new contribution to the field is the
addition of the inter-finger joint dependency constraints described later. No reference to the use
of such constraints was mentioned in any of the hand modeling literature surveyed [2, 3],
although it was mentioned in the medical literature [4, 5] that dependencies did exist between
joints on neighboring fingers. However, no quantitative relationships or data was found.
Also, while the Denavit-Hartenburg conventions have been applied in biomechanical modeling
[6], no reference to their use in hand modeling was found. This contribution, while not likely
original, was developed independently.
Project Implementation
The model described in this project has been implemented as a C++ program compiled using
Optima’s Power++ Compiler for the Windows95/98/NT platform. The model is rendered using
OpenGL. Each coordinate frame in the model and its associated finger bone is represented as an
instance of the "Thing” class. All “Things” are represented internally in a hierarchical structure
where all accesses are made through the root “Thing”. In this case, that would be the base of the
hand.
3
Model Description and Background
This section will describe the components of the human hand model developed for this project.
Each subsection will first introduce the relevant background material before addressing the
relation of that material to the construction of the model.
Skeletal Anatomy of the Human Hand
We will discuss the skeletal anatomy of the human hand in order to answer the question posed
earlier about the movable joints of the hand. The definition of movable joints is taken from a
number of sources, including both people who work in the field of gesture recognition [2, 3] and
those in the medical community [4, 5].
The human hand skeleton is composed of 27 bones. These bones can be divided into three
groups:
•
•
•
eight carpals
five metacarpals
phalanges
The carpals are found in the wrist, the metacarpals in the palm, and the phalanges are the finger
bones. Joints between these bones will vary in the number of degrees-of-freedom they possess.
The number of degrees-of-freedom at a joint is the number of axes-of-rotation at that joint. The
following figure [6] illustrates the bones mentioned above, their joints, and the number of
degrees-of-freedom (indicated by DoF) at each joint:
4
Figure 1 Skeletal Anatomy of the Human Hand
We will refer to the notation in the figure above when discussing a particular joint. For example,
the “PP joint” will refer to the first knuckle of a finger. It is necessary, at this stage, to introduce
the medical terminology that defines movement in the hand. Flexion refers to the movement of
fingers towards the palm. Extension refers to the movement of the fingers away from the palm.
Abduction refers to movement of the fingers away from the plane that divides the hand between
the middle and ring fingers. Adduction refers to the movement of fingers towards this plane. The
same terminology is used in relation to the thumb.
All the fingers have the same essential structure. The MC joint exhibits 1 DoF, the PP exhibits 2,
the MP exhibits 1, and the DP exhibits 1. Each of these joints exhibit flexion and extension.
However, only the PP joint of the fingers exhibits abduction and adduction. Note that all of this
motion is being generated at the metacarpals and phalanges. There is a certain degree of motion
5
generated in the carpals (wrist bones). However, this motion is quite limited and is not being
modeled.
The thumb has a more complex structure. The MC joint exhibits 3 degrees-of-freedom, the PP
exhibits 2, and the DP exhibits 1. The MC joint exhibits flexion/extension, abduction/adduction,
and also rotation about the axis of the metacarpal. The PP joint is similar to the PP joint of the
fingers in that it exhibits flexion/extension and abduction/adduction. The DP joint exhibits
flexion/extension.
In total, the hand model has 26 degrees-of-freedom. We shall soon see this number reduced as
constraints are applied to the model.
Developing Model Coordinate Frames using Denavit-Hartenburg
Denavit and Hartenburg developed a systematic method for assigning coordinate frames to links
in a kinematic chain of links. After this method has been applied, it is possible to apply standard
transformations to move between coordinate frames. These transformations are described by
only four parameters:
Parameter
Joint angle
Link length
Joint distance
Link twist angle
Symbol
α
l
d
θ
These four parameters form a transformation matrix, A, which is developed as follows:
A = Rot( z, θ ) * Trans( 0, 0, d ) * Trans( l, 0, 0 ) * Rot( x, α )
A=




cos(θ)
sin(θ)
0
0
-sin(θ) * cos(α)
cos(θ) * cos(α)
sin(α)
0
sin(θ) * sin(α)
-cos(θ) * sin(α)
cos(α)
0
l * cos(θ)
l * sin(θ)
d
1




Thus, each coordinate frame will be related to the previous one by the above matrix filled in with
the four parameters for that frame. The selection of these parameters must be done in accordance
with the Denavit-Hartenburg convention. The salient points of this process that relate to the
creation of the model in this paper will now be described. However, for a full description refer to
[1].
The Z axis should be assigned to the axis of rotation.
The X axis should be aligned with the common normal of the Z axes of the frame under
consideration and the previous frame (from which we are transforming). This will normally
mean that the X axis is aligned with the body of the link.
The Y axis should be assigned as the cross product of the X and Z axes.
For all coordinate frames in the hand model, the only model parameter the will be modified after
initial model creation will be the θ parameter. In order to model joints with more then one
degree-of-freedom, multiple coordinate frames are required. Thus, the first such coordinate
6
frame for a particular joint (first meaning closer to the wrist) will have an “l” parameter, or link
length, of 0. Only the final coordinate frame associated with a particular link will have a link
length that reflects the actual length of the attached phalange. For example, the PP joint of the
fingers is modeled with two coordinate frames. The first has l = 0, whereas the second has l equal
to the length of the proximal phalange for that finger.
We will now present a diagram that illustrates the assignment of coordinate frames to a finger.
As mentioned before, all fingers have the same essential structure so the following diagram is
applied to all fingers in the model.
Figure 2 Assignment of coordinate frames to finger joints
7
As shown above, the coordinate frame attached to the MC joint is transformed from the world
coordinate system by simple translation and rotation.
The assignment of coordinate frames to the joints of the thumb is similar to the above assignment.
However, it is made much more complex due to the three degrees-of-freedom exhibited at the
MC joint. A total of five coordinate frames were required to model this joint and the attached
metacarpal.
Joint Dependency Constraints
We may divide the constraints placed on joint angles into three groups:
Static Constraints
Intra-Finger Joint Dependency Constraints
Inter-Finger Joint Dependency Constraints
We will discuss each of these in turn.
Static Constraints
Joints in the human hand have only a restricted range of motion. However, defining this range of
motion is made somewhat ambiguous by the fact that when extra pressure is applied the range of
motion can increase beyond normal bounds. For an extension movement, this is known as
hyperextension. Also, the range of motion is dependent upon the individual.
The normal range of motion of human hand joints corresponds to static constraints on joint angles
in the model. These constraints are just limits on the values that the θ parameters can take. The
values used are listed below [3]. Note that flexion/extension is indicated by f/e and
abduction/adduction is indicated by (a/a).
-10° < PP Joint (f/e) < 90°
-15° < PP Limit(a/a) < 15°
-20° < MC Limit(f/e) < 15° ( for the ring and pinky fingers )
The final constraint applies only to the ring and pinky fingers. The normal range of motion for
that same joint in the middle and index fingers is extremely limited [4].
Intra-Finger Joint Dependency Constraints
When the PP joint of a finger flexes or extends, there is often a corresponding rotation in the MP
joint of that same finger. This is due to the fact that some of the same underlying musculature is
involved in both movements and they are therefore often naturally coupled. Studies of the hand’s
anatomy, coupled with empirical studies [2] show a near linear relationship between these joint
rotations:
MP Joint (f/e) = 2 * PP Joint (f/e)
Applying this constraint on the model reduces the total number of degrees-of-freedom by four,
leaving us with 22 degrees-of-freedom.
A similar constraint is found to apply between the MP joint and the DP joint:
8
DP Joint (f/e) = 2/3 * MP Joint (f/e)
Applying this constraint reduces the number of degrees-of-freedom, again by four leaving 18
degrees-of-freedom. The above two constraints applied together will produce a natural-looking
bending of the fingers when the PP joint is flexed.
One further, intra-finger constraint that can be applied relates the range of abduction/adduction
that the PP joint of the fingers can exhibit at varying levels of flexion/extension. If the PP joint is
neither flexed nor extended then the range of abduction/adduction is maximized. If the PP joint if
fully flexed then no abduction/adduction is possible. The following relation has been developed
to relate the range of abduction/adduction at the PP joint to the amount of flexion at that joint [2]:
PP Joint(a/a) = ( PP Joint(f/e) / 90° ) * ( PP Converge - PP Limit(a/a) ) - PP Limit(a/a)
where,
PP Converge = Flexion angle where maximum range of abduction/adduction is observed
= 0° (in model)
PP Limit(a/a) = Maximum abduction/adduction angle
(observed at a flexion angle of PP Converge)
= 15° (in model)
The application of this constraint does not alter the number of degrees-of-freedom in the model.
However, in relation to our intended application, it does reduce the effective joint-space.
Inter-Finger Joint Dependency Constraints
It is difficult, if not impossible, to move certain fingers independently when they are in a certain
configuration. This can be observed by attempting to fully flex the PP joint of one’s ring finger
without moving the neighboring fingers. This tendency is modeled here by specifying a minimal
angular distance that can be maintained between one particular joint on one finger, and another
joint on some other finger (or possibly the same finger). To mimic the tendency of neighboring
fingers to flex together, this model includes a minimal angular distance between the flexion angle
of each PP joint (f/e) and the flexion angle of the same joint on the neighboring finger(s). For
example, constraints on the ring finger’s PP joint would yield the following relationships:
| Ring PP Joint (f/e) - Pinky PP Joint (f/e) | < 45°
| Ring PP Joint (f/e) - Middle PP Joint (f/e) | < 40°
Note that these numbers were not derived from any anatomical or experimental studies. They are
merely first guesses at likely values for these parameters.
In the running demonstration of this model, when a finger’s PP joint reaches beyond the
maximum angular distance between its flexion angle and the neighboring finger’s PP joint flexion
angle, that neighboring joint is made to move so that maximum distance is maintained.
9
This constraint, besides creating a natural-looking effect, also reduces the effective space of
model parameters that is permitted.
10
Results and Conclusions
Results
It is difficult to assess how successful the creation of this model has been with regards to the
original project goal. The reason for this is the fact that our original goal involved the use of this
model as an embedded component in a gesture recognition system. However, the model is
currently the only existing component of this system. Therefore it is impossible to accurately
ascertain how closely this model has come to achieving its desired purpose.
This being said, the model has sufficient complexity to match virtually any conceivable hand
posture. Also, the addition of the constraints discussed significantly reduces both the total
number of degrees-of-freedom in the model (from 26 to 18), and the range of motion for some of
the remaining movable joints.
It is interesting to note that a tradeoff exists between these two points just mentioned. A model
with sufficient complexity to match any real hand posture would exhibit a relatively large number
of degrees-of-freedom and allow a large range of motion for its joints. On the other hand, the
parameter space of such a model would be significantly larger then a model that exhibited fewer
degrees-of-freedom and applied constraints on range of motion. But, this more constrained
model would not be able to achieve the full gallery of hand postures that were possible with the
other model. For example, our constrained model cannot flex its PP joints without flexing its MP
and DP joints. This would preclude hand postures such as is illustrated in the following figure:
Figure 3 An impossible hand posture for the constrained model
Conclusions
A successful model of the human hand has been developed in the course of this project. The
model is articulated in the same manner as the actual human hand and employs many of the same
constraints on its motion.
However, the model is by no means complete. The constraints applied are merely rough
approximations of the kind of restrictions that are applied to the real human hand’s range of
motion by its musculature. In order to create a more accurate model, it may be necessary to
increase the depth of the modeling to include the muscular layer. This would also allow us to
model subtler forms of movement that are not readily tractable in a pure kinematic model such as
is employed here.
11
Another required addition to this model would be some sort of calibration. Currently, the
software includes hard-coded values for the geometry and positioning of the joints and fingers.
Some manner of automatic calibration utility is required to “mold” the model to match the
dimensions of a particular person’s hand.
In addition, the rendered model is currently comprised of cylinders and spheres. It would greatly
improve the model if it were to be covered instead with a surface that more closely matched the
actual surface geometry of the human hand. Furthermore, a full surface model would also
include bending and wrinkling of the skin.
There is much further work to be done. However, it is felt that this model will serve as a good
first step in the development of the gesture recognition system it was intended for.
12
References
[1] Stone, Henry W., "Kinematic Modeling, Identification, and Control of Robotic Manipulators",
Kluwer Academic, 1987, pp. 10-14.
[2] "Fifth International Conference on Computer Vision", IEEE Computer Society Press, 1995,
pp. 668.
[3] "IEEE Transactions on Pattern Recognition and Machine Intelligence", IEEE Computer
Soociety Press, Vol. 19, Number 7, July 1997, pp. 681-682.
[4] Norkin, Cynthia C., Levangie, Pamela K., "Joint Structure & Function: A Comprehensive
Analysis", 2nd Ed., F.A. Davis, Philedelphia, 1992, pp. 272-290.
[5] Brand, Paul W., Hollister, Anne., "Clinical Mechanics of the Hand", 2nd Ed., Mosby Year
Book, St. Louis, 1993.
[6] http://www.cchs.su.edu.au/Academic/BIO/biomech/abc/p27.html
13