Development of a glove-type input device with the minimum number of
sensors for Japanese finger spelling
Y Tabata1, T Kuroda2, K Okamoto3
1
Department of Radiation Technology, Kyoto College of Medical Science,
1-3 Imakita, Oyama-higashi, Sonobe, Nantan, JAPAN
2,3
Department of Medical Informatics, Kyoto University Hospital,
54 Kawahara-cho, Shogo-in, Sakyo-ku, Kyoto, JAPAN
1
[email protected], [email protected], [email protected]
www.kyoto-msc.jp, 2,3www.kuhp.kyoto-u.ac.jp
1
ABSTRACT
A glove-type input device, which can measure hand postures of human beings directly, is one
of essential device to develop Virtual Reality environment. The authors have been developing
a data-glove which would be able to capture hand postures according to user’s demand with
the minimum number of sensors. Our previous research estimated the data-glove with six
sensors could measure all hand postures for Japanese Finger spellings. Thus, this paper
proposes a prototype with six sensors and evaluate whether the prototype glove sensor can
distinguish all hand postures of Japanese Finger spellings. This evaluation indicated that dataglove with fewer sensors than conventional number of sensors could distinguish hand postures
exactly.
1. INTRODUCTION
The hearing impaired have the communication barrier, which is still a serious problem, in their social lives.
Because there are few situations it could be understood, while the most familiar method of communication
for the hearing impaired is sign language.
Though a lot of information recently are provided visually by rapid progress of information technologies,
for example, images, e-mails, subtitles for pictures and so on, the hearing impaired seem to feel some
difficulties to get information via the written phonetic Japanese, As their mother tongue, Japanese Sign
Language (JSL), which has a quite different grammar system.
First of all, these systems need to have input devices to capture each word of sign language, and there are
two major methods to capture hand postures. One is the vision-based method, and the other is the glovebased method (T Stamer 1998). The vision-based method frees its users from a hassle wearing devices, but it
is still challenging because the vision-based method has problems as capturing not only hand postures but
also whole body motions, and besides, in the case of postures of small hands being affected by lighting
conditions.
On the other hand, the glove-based method can measure hand postures directly. However, the price of the
gloves available for sign capturing is still expensive, then it is necessary to develop an inexpensive glovetype input device (data-glove) suitable to measure sign language.
So the authors proposed the method to define the minimum number of sensors to capture a set of the
given hand postures. The previous research estimated that a data-glove equipped with six low-resolution
sensors could measure whole Japanese Sign Language (JSL) finger spelling (Tabata et al 2010). The lowresolution sensor itself may be produced at low cost, then the glove consisting of the limited number of
sensors may also reduce the whole expense. Thus, if the glove designed under this method is confirmed
feasible by experiments, it could open up the huge possibility for glove-based systems.
The purpose of this paper is to make a prototype of data-glove with six sensors and to evaluate whether
the prototype can distinguish each hand posture in JSL finger spelling given by users.
Proc. 9th Intl Conf. Disability, Virtual Reality & Associated Technologies
Laval, France, 10–12 Sept. 2012
2012 ICDVRAT; ISBN 978-0-7049-1545-9
305
2. RELATED WORKS
2.1
Data-glove
Hands are the most useful tools for us to deal with a lot of things in our everyday environment. Therefore,
many researchers have been developing technologies for manipulating our surroundings by using human
hands (Dipietro et al 2008). Above all, glove-type input device is the most popular as data acquisition system
for hand movements. And the glove-type input device, which is called data-glove, is one of essential devices
to develop Virtual Reality environment.
After Zimmerman had developed Data Glove (Zimmerman, 1982), a number of data-gloves has been
proposed and developed over and over. Most of the data-gloves use various sensors such as optical fibers,
piezo registers, carbon ink, magnetic, etc, in order to measure finger joint angles. In addition, some datagloves, for example Pinch Glove, are equipped with contact sensors to acquire the data of contact states of
fingertips. The authors also have developed a data-glove called “StrinGlove” ,which equips 24 inductcoders
and nine contact sensors, to realize the consumer price of glove sensor (Kuroda 2004).
These data-gloves have been used either as a pure motion capturing device or as a command input
interface. In the case of command input, the data-glove should capture just "digital" hand postures. Gesture
recognition systems and sign translators are other typical and conventional examples. In these applications,
data-glove are adapted to obtain some hand postures or hand motions which mean a gesture (Y.Lee et al
2007). Therefore, data-gloves have been used as a input device like keyboards and mice in several kinds of
research fields (K.Tsukada 2004).
2.2
Sign Notation
Sign language is the most important communication method for the hearing impaired and has the most
sophisticated coding system of human motions. Therefore, when we think about hand postures, we may refer
sign notation codes as a gold standard of hand postures.
Stokoe et al (1965) studied Sign language with a linguistic view for the first time in the world, and his
group claimed that sign words consisted of three elements; "Dez" (hand posture), "Tab" (location), and "Sig"
(movement) and have original characters on each element. In this way, every hand posture has its own
original character, but it is difficult to form exact postures of each finger.
The Stokoe's notation characters were used in many researches of sign linguistics, and recently new
notation characters have been developed by referring the Stokoe's notation (Kanda 2010).HamNoSys
developed by German researchers was a system of sign notation characters for computers and denoted hand
postures as a set of finger postures (Hanke 2004). Kurokawa (1992) also developed the method to express
hand postures by using a set of finger postures from researches of non-verbal human computer interaction.
The features of these systems are to express a hand posture by using combination of contacts of fingers
and bending angles of each joint, and to classify hand postures by subjective point of view. Therefore, these
systems don’t make the slightly bending position have its own code, as these systems must be influenced by
human perception on hand postures. Then, an analysis reports that the bending angle of each joint is
classified into three levels (full bend, weak bend and stretch).
3. EVALUATION
3.1
Prototype
In the proposed algorithm shown in Figure.1, a designer subjectively denotes a set of target hand postures as
a set of sensors (code-sensor relation table). Then, the set cover of the table with minimum number of sets
provides the minimum number of sensors. Figure.2 shows the target hand postures (28 hand postures of JSL
finger spelling). Table.1 shows the obtained set cover, and figure.3 shows the estimated data-glove for JSL
finger spelling (left). The sensor consists of five two-bits bend sensors and one one-bit contact sensor.
The authors have utilized the existing StrinGlove™ as the prototype, and evaluated the glove decorated
with the minimum number of sensors. And three subjects expressed the 28 hand postures this time, and the
given postures are measured and investigated by the prototype shown in Figure.3. The threshold values,
which were used to output two-bits and one-bit, were manually-set at approximately regular intervals. Three
subjects are two females and one male.
306
Proc. 9th Intl Conf. Disability, Virtual Reality & Associated Technologies
Laval, France, 10–12 Sept. 2012
2012 ICDVRAT; ISBN 978-0-7049-1545-9
Figure 1. Overview of our proposed method.
Figure 2. Target hand posture (28 hand posture of JSL Finger spelling).
Table1. The obtained set cover.
Proc. 9th Intl Conf. Disability, Virtual Reality & Associated Technologies
Laval, France, 10–12 Sept. 2012
2012 ICDVRAT; ISBN 978-0-7049-1545-9
307
Figure 3. Sensor Layout(left) and the prototype(right).
3.2
Results
The prototype distinguished 82% of the given hand postures, compared with the difference between sensor
values of the obtained set cover and the measured values of the prototype. Table 2 shows the causes of the
errors. The two sensors on ring and pinkie, no.5 and no.6, were the main cause of errors. In addition, the
prototype could not distinguish some hand postures; “2” and “3”, “2” and “U”,“CHI” and “TSU”, “A” and
“TA”.
Table.2: Causes of errors.
3.3
Sensor
ratio
no.1
8%
no.2
13%
no.3
10%
no.4
13%
no.5
33%
no.6
23%
Discussion
The sensor no.5 is the bending sensor for measuring angle of distal interphalangeal joint (DIJ) of pinkie. As
the pinkie of the prototype was little bit loose, the slip between glove and pinkie might cause a lot of errors.
To overcome the problem the glove should be made of more stretchable material to fit the glove tightly. The
same problem happened in the case of sensor no.2 and no.4, which are the bending sensors for measuring the
distance between two fingers.
The sensor no.6 is a contact sensor used for measuring the contact between middle and ring fingers. A
proximity sensor was utilized as the contact sensor based on magnetic coupling on the prototype. Thus the
sensor sometimes misfires even when the distance between the two fingers is not zero. Using a simple switch
instead of the proximity sensor may overcome the problem. In addition, it is important to think about also
arrangement of a contact sensor to measure the contact between middle and ring accurately. The senor no.6, a
set of contact sensor, is put on the sides in opposition each other between middle and ring fingers shown in
Figure.3 (right). Thus, the set of contact sensor easily fall from the glove comparing with the other sensors,
since they touch each other and catch on the glove fabrics by accident when subjects express hand postures.
To overcome these problems, it is also one of the necessary factors to downsize a contact sensor, thought the
progress of sensing technology is essential factor to measure a contact state strictly.
The results indicated that the prototype could not distinguish some hand postures, and Table.3 shows
examples of the errors of distinction. Figures are the measured values, and figures in parentheses are the
correct values in Table.3.
One of the reasons of errors would be simple error of sensor no.4, no.5 and no.6 with high probability.
And besides, error of only one sensor causes distinction error in each combination sample as shown in
Table.3. Therefore, the prototype has importance to measure hand postures accurately. On the other hand, to
add a sensor to the prototype may overcome this problem of not distinguishing these hand postures. For
308
Proc. 9th Intl Conf. Disability, Virtual Reality & Associated Technologies
Laval, France, 10–12 Sept. 2012
2012 ICDVRAT; ISBN 978-0-7049-1545-9
example, hand posture “2” and “3” have different values on sensor no.5, however, the data of these two were
equal to each other this time. Thus, one additional sensor on ring finger may distinguish these hand posture.
Therefore, the prototype with fewer sensors than conventional number of sensors will be able to distinguish
hand postures exactly. On the other hand, the result shows that the authors found the certain positions of
sensor getting error, then the prototype could measure clearly by changing sensor positions of weak reflection,
thought it may be difficult to develop the prototype with six sensors.
Another reason is whether each designer can denote target hand postures strictly. Because hand postures
are denoted subjectively, they may be given in different ways by different designers. However, this
evaluation confirmed that all subjects folded/stretched according to what the table defines, so the difference
of subject may not become an indistinctive reason. But it would be necessary to confirm whether the denoted
postures are different with each person under an increase of subjects.
Table.3: Distinction errors of hand postures.
The other reason is that the subjects may have expressed hand postures in their own way. A subject told that
this hand posture of “A” was different from the hand posture which he always expresses in his daily life. The
prototype has been developed to capture “digital” hand postures. Therefore, the slight different between
target hand postures and expressed hand postures may give an influence to distinguish these hand postures.
The threshold processing may give an influence with the distinction of hand postures, because the
threshold processing classifies data, which bending and contact sensors measure first, into two-bit values and
one-bit values, and each sensor of the prototype finally outputs the classified values. The major parameter in
the threshold process is a selection of the threshold value. There are many methods to select the proper
threshold values like histogram shape-based thresholding method in image processing (M.Sezgin et al 2004),
but the prototype utilizes the manually-set threshold values at approximately regular intervals. As the
distinction rate of the prototype was 82%, the prototype would use proper thresholds. However, the sensor
values of the obtained set cover table were determined by a designer’s subjectivity. Thus, it would be
important idea to decide the threshold values in subjective view of a designer. That is, it might be effective
method that it compares the measured values of the prototype with the determined values by subjective point
of view and chooses the thresholds by using the comparison result. Therefore, the authors will conduct
additional experiment about the method to choose the threshold values.
The prototype with only six sensors did not have enough processing function to distinguish 28 hand
postures of JSL finger spelling. But, it would be necessary for the prototype to distinguish 28 hand postures
by using the values of only six sensors, as it cannot have large number of sensors like the conventional dataglove. And so, the distinction rate would increase if it is possible that the prototype has the function to
estimate the correct sensor values with the six sensors.
The result tells that a single sensor measurement error becomes fatal error under the condition with the
optimized data-glove as proposed. Therefore, to provide the redundancy to ease the sensor error problem will
be the discussion for better manufacturing of data-glove. To make data-glove designing process more
effective, a firm algorithm to provide sensor redundancy is indispensable in the future.
Finally, the prototype has a possibility to be used as a command-based input interface for VR system
when some postures of JSL are applied as commands, even though it has been developed to measure hand
postures of JSL finger spellings. Moreover, as the prototype is developed by using our proposed method to
find the minimum number of sensors, the prototype in final version indicates some possibilities to make up a
special data-glove for each user. So, this research will be able to make a contribution to data-glove being
popular in several kinds of fields as one of glove-type input devices.
Proc. 9th Intl Conf. Disability, Virtual Reality & Associated Technologies
Laval, France, 10–12 Sept. 2012
2012 ICDVRAT; ISBN 978-0-7049-1545-9
309
4. CONCLUSIONS
The aim of this paper was to make a prototype of data-glove with six sensors and to evaluate whether the
prototype can distinguish each hand posture in JSL finger spelling given by users. The developed prototype
outputs the bending values and the contact values with six sensors, which consists of five two-bits bend
sensors and a one-bit contact sensor, and utilized the existing StringGlove.
The prototype was evaluated in this experiment. The prototype distinguished 82% of the given hand
postures, but the prototype could not distinguish some hand postures. One of the reasons, that it fails to
distinguish them, was simple error of sensors. The two sensors on ring and pinkie were the main cause of
errors. Therefore the evaluation indicates that the improvement of these sensors would reduce the errors of
measurement. Moreover, in this experiment, some refinements were found to improve the prototype. By
adding the refinements to the prototype, the result indicated that a data-glove with fewer sensors than
conventional number of sensors will be able to distinguish hand postures exactly, though it would be difficult
to develop the data-glove with only six sensors. However it would be necessary to conduct additional
evaluations, to achieve the development of the prototype with fewer sensors to distinguish them exactly.
This evaluation may clear that a single sensor measurement error becomes fatal error under the condition
with the optimized data-glove as proposed. Therefore, in the future, a method to provide sensor redundancy
is indispensable in order to make data-glove designing process.
Lastly, the data-glove with minimum numbers of sensors would be capable to use as command-based
input interface, though the authors have developed the data-glove for JSL Finger spelling, The final version
of the prototype could clear that our proposed method to find the minimum number of sensors have an
effective method to develop data-glove.
Acknowledgements: The authors would like to thank Dr. Shinobu Kawagishi for their support for defining
hand posture set of JSL. The authors also would like to thank Teiken Limited, Fujita Corp. and AMITEQ
Corp. for their continuous support to develop StrinGlove®.
5. REFERENCES
L. Dipietro, A.M. Sabatini and P. Dario (2008), A Survey of Glove Based Systems and Their Applications,
IEEE Trans. Man. Cyber., 38, 4, pp.461-482.
T Hanke (2004) HamNoSys - Representing Sign Language Data in Language Resources and Language
Processing Contexts, Proc. LREC, pp.1-6.
K.Kanda (2010), A Study of Linguistic Characteristics of Japanese Sign Language, Fukumura Press, Tokyo,
Japanese.
T Kuroda, Y Tabata, A Goto, H Ikuta and M.Murakami (2004), Consumer Price Data-glove for Sign
Language Recognition , Proc.5th Intl Conf. Disability, Virutal Reality & Assoc. Thech., pp. 253-258.
T. Kurokawa (1992) Gesture Coding and a Gesture Dictionary for a Nonverbal Interface, IEICE Trans. Fund.
Elect., 75, pp.112-121.
Y Lee, S Min, H Yang and K Jung (2007), Motion Sensitive Glove-based Korean Finger spelling Tutor,
International Conference on Convergence Information Technology, pp.1674-1677.
M.Sezgin and B.Sankur(2004), Survey over image thresholding techniques and quantitative performance
evaluation, Journal of Electronic Imaging ,13,1, pp.146-165
T Stamer and A.Pentland (1998), Real-Time American Sign Language Recognition Using Desk and Werable
Computer Based Video, IEEE Transactions on Pattern Analysis and Machine Intelligence, 20, 12,
pp.1371-1375.
W.C. Stokoe, D.C.Casterline and C.G. Croneberg (1965) A Dictionary of American Sign Language on
Linguistic Principles, Gallaudet College Press, Washington DC.
Y Tabata, T Kuroda, K Okamoto(2010), Basic study toward automatic generation of glove-type command
input device with optimal number of sensors, Proc. 8th Intl Conf. on Disability, Virtual Reality and Assoc.
Technologies, pp. 283-289.
K Tsukada and M Yasumura (2004), Ubi-Finger: a Simple Gesture Input Device for Mobile and Ubiquitous
Environment, Journal of Asian Information, Science and Life (AISL), 2, 2, pp.111-120
T.G. Zimmerman, J. Lanier, C. Blanchard, S. Bryson and Y. Harvill (1987) A Hand Gesture Interface
Device, Proc. SIGCHI, pp.189-192.
310
Proc. 9th Intl Conf. Disability, Virtual Reality & Associated Technologies
Laval, France, 10–12 Sept. 2012
2012 ICDVRAT; ISBN 978-0-7049-1545-9