ePet: When Cellular Phone Learns to
Recognize Its Owner
Mohammad Tanviruzzaman, Sheikh Iqbal Ahamed, Chowdhury Sharif Hasan, Casey O’brien
Marquette University, Milwaukee, Wisconsin, USA
{mtanviru, iq, chasan}@mscs.mu.edu, [email protected]
most of the users. The users eventually choose a few easy to
remember passwords for all their tasks and thus the level of
protection decreases significantly. Though “strong passwords”
can provide very high level of security, most of the time they end
up being scratched on a notebook and are kept quite unprotected.
ABSTRACT
In this paper an adaptive solution to secure the authentication
process of cellular phones has been proposed. Gait and location
tracks of the owner are used as the metrics for authentication. The
cellular phone is envisioned to become as adaptive as a pet animal
of the owner. The cellular phone learns various intrinsic attributes
of the owner like his voice, face, hand and fingerprint geometry
and interesting patterns in the owner’s daily life and remembers
those to continually check against any anomalous behavior that
may occur due to the stealing of the phone. The checking is done
level wise. Higher level of authentication is more stringent. Only
when the cellular phone recognizes significant anomaly in a lower
level, it goes one level up in the security hierarchy. The iPhone’s
accelerometer and A-GPS module have been utilized to record
gait and location signatures. A fast and memory efficient variation
of Dynamic Time Warping (DTW) algorithm called FastDTW has
been used to compute the similarity score between gait samples.
Behavioral or physiological attributes of the owner can be
recorded and can be used to authenticate the owner’s identity at a
later time. Physiological attributes include fingerprint and iris
recognition [18], voice pattern, face and hand geometry [19], etc.
and behavioral attributes include, file system and network activity
[16], etc. Presently, some of the cellular phones like Pantech PG6200 are equipped with integrated fingerprint reader. Some cell
phones like Vodaphone 904SH have face recognition capabilities.
Although biometrics helps to lessen the memory problems and
also is a less obtrusive method of authentication than password or
PIN, it is not a panacea. Fingerprints and hand geometry are easily
recreated in plastic. Voice can be mimicked easily or it can be
distorted due to cold and face can be reproduced or it can have a
scar on it. There are other problems with physiological biometrics
like older adults with impoverished skin may have difficulty with
enrolling and verifying their fingerprints. People might be
threatened by the fact that the authentication server is holding
information about their very personal attributes. Such fears have
been strengthened in the UK with the loss of two computer discs
that contained 25 million personal data records [4]. Behavioral
biometrics like signature, voice or gait is vulnerable to spoof
attacks [6].
Categories and Subject Descriptors
K.6.5 [Management of Computing and Information Systems]:
Security and Protection – Authentication
General Terms
Security
Keywords
Authentication, Cellular Phone, Security, Biometrics, Gait,
Location Track, iPhone, Accelerometer, GPS
The authentication schemes for the cellular phones are of two
types: memory based and attribute or biometrics based. In the
present world with so much surveillance on people’s lives and
advanced attack-scenarios the most secured authentication metric
can be the user himself or in other words, the collection of the
intrinsic attributes of a user.
1. INTRODUCTION
Wireless Miniaturization techniques like tiny but powerful IC or
nanotechnology have brought with it the rapid increase in the
performance of cell phones. Their increased performance has
enabled them to run many critical applications like m-banking
[25] and to store crucial data like financial records. Thus security
for cellular phones has become a dire necessity.
In the light of this statement we propose a novel solution to the
authentication problems for cellular phones. The solution has
similarity with multimodal biometrics based authentication
schemes but it has its unique nuances. In our solution we envision
the cellular phone as if it were a pet animal of the user which will
know its owner better with time [4] and thus will be able to
distinguish if someone else is carrying or using it other than its
owner and thus can lock itself as needed. We not only utilize the
physiological and behavioral biometrics for authentication, but
also the data related to the user’s surroundings. The cellular phone
records samples of various significant characteristics of the user
like his gait-pattern, the places he usually visits, his voice, etc. It
also records fingerprint and face pattern. The authentication
process is stratified. The upper levels provide stronger security.
The user can keep the response to the memory based challenge to
A mobile phone is stolen every three minutes in the UK [14].
Presently, PIN codes or passwords are used for the security of
these devices. These mechanisms cause “memory overload” for
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that
copies bear this notice and the full citation on the first page. To copy
otherwise, or republish, to post on servers or to redistribute to lists,
requires prior specific permission and/or a fee.
SafeConfig’09, November 9, 2009, Chicago, Illinois, USA.
Copyright 2009 ACM 978-1-60558-778-3/09/11…$10.00.
13
So, if someone mimics the owner’s walking style pretty well, still
he will fail to meet the strict similarity requirement required by
the cellular phone in a place never or rarely visited by the owner.
a secured area and he can depend on the biometrics and
environmental metrics based checking for security most of the
time. Our contributions:
•
A novel and adaptive solution to the cellular phone security
is introduced.
•
The solution eradicates the need for the user to continually
keep difficult passwords in memory.
•
The authentication process in our solution is silent and less
intrusive most of the time, i.e., the authentication process
runs continuously in the background until a higher level of
security is needed.
•
We have developed the solution using iPhone. We have
shown that using iPhone’s current capabilities such an
adaptive solution to security can be achieved.
3. RELATED WORK
2. ARCHITECTURE
In [10] personal gestures have been recognized by a single
three-axis accelerometer. It avoids statistical methods and
instead requires a single training sample for gesture
recognition. It has been implemented on smart phones with an
accuracy of 98.6%. In [8] gait is collected from the acceleration
signals collected from sensors attached to the person’s body.
This gait data is used to recognize persons. It has been shown
that impostors who know their closest person in the database or
the genders of the users can be a threat to gait based
authentication. In [7] location tracks of users have been used to
identify their homes. The significance of this work in our
architecture is that we shall find out the familiar or frequently
visited places of a user in a similar way. In [9] a three-axis
accelerometer on the waist belt is used to detect acceleration of
body movement. In [13] a fuss-free gait analyzer based on a
single three-axis accelerometer attached to a cellular phone for
health care and presence services is proposed. In [1] gait signal
from a three-dimensional accelerometer is used to identify the
users of portable devices. Correlation, frequency domain and
data distribution statistics are used to identify test subjects who
walked with fast, normal and slow walking speed.
In Figure 1 the architecture has been depicted diagrammatically.
In our proposed architecture, there are three modules: feature
extractor module, matching module and decision module. Feature
extractor module takes data from cellular phone sensors and
extracts interesting patterns from the raw data and saves it as a
template. At a later time the cellular phone sends the newly found
cycle data along with the template to the matching module. The
decision module receives N scores from N primary level metrics.
Each of the N metrics may have a different threshold acceptance
value. Thus the decision module compares the N scores with their
corresponding threshold values. If any of the N comparisons ends
up in being less than the threshold value, the security checking
gets more stringent or the checking may go one level up. Only
when all of the N values are greater or equal to the corresponding
threshold values the carrier is accepted as the owner for the time
being. A cellular phone performs this authentication process
periodically.
In [14] gait patterns are extracted from a physical device
attached to the lower leg. Combination of vertical, forwardbackward, and sideways motion of the lower leg is used for
authentication. In [6] various scenarios that are possible in
multimodal biometric systems, the levels of fusion that are
plausible and the integration strategies that can be adopted to
consolidate information have been addressed. A couple of
multimodal systems in the literature have also been discussed
there. In [11] effects of a number of real-world factors like
different shoes, extra load, and the natural variation over days
on the gait are addressed. It is shown that these factors may
affect the gait of an individual so much that false rejection may
occur. In [12] an approach on identifying users based on threedimensional gait acceleration signal characteristics acquired by
a portable accelerometer attached to the center of the user’s
waist is analyzed. Recognition is based on the general idea of
template matching through dynamic time warping algorithm.
The rest of the paper is organized as follows. Section 2 depicts
overall architecture of the system using a comprehensive diagram.
Section 3 focuses on the related works. Section 4 walks through
the necessary details of iPhone sensors used in our
implementation. We present the algorithms in Section 5. This
section also contains results of our experiments. Future works and
avenues for improvements are discussed in Section 6. Finally, in
Section 7 we conclude the paper.
In [4] an idea of a fictitious biometric device is conceived
which is initially imprinted with the fixed biometric properties
of its owner, and is then regularly updated with the fluid
biometric properties of the owner. In [5] an empirical mode
decomposition based feature extraction technique for the
classification of unsupervised walking activities from
accelerometer data is described. Acceleration data is collected
from a single waist-mounted tri-axial accelerometer. In [3] a
distributed multimodal biometric verification system is
described. It utilizes Java Remote Method invocation
framework to link Server and Client objects. In [2] a
multimodal authentication system called HUMABIO is
described. It utilizes micro and nano-sensors to capture
biometrics like face, speech, gait and seat-based
anthropometrics.
Normal and anomalous usage of battery-powered portable
devices is distinguished using file-system and network access
Figure 1. System Architecture
14
patterns in [16]. The system is found to be 95% accurate in
recognizing attacks from normal usage. In [17] a method to
authenticate users whilst typing text messages, using two
keystroke characteristics, the inter-keystroke latency and holdtime is described. [18] presents an iris pattern based
authentication for smart phones. In [19] a multimodal biometric
system based on face and hand images captured by a cell phone
are presented. The best accuracy of up to 99.82% has been
achieved for the model combining 8 eye, 12 mouth and 9 hand
features.
work we have used the gait data and location tracks. The iPhone
extracts the gait cycle pattern from the accelerometer data and
also finds out the owner’s familiar places using his location
tracks. The phone periodically checks if the person carrying it is
the owner by matching current gait pattern with the saved gait
template and it also checks if it is in a familiar place of the owner.
If the phone finds out that it is in a familiar place, it will accept a
rather average gait matching score. But whenever the phone finds
it in an unfamiliar place, it requires a very exact match of the gait
patterns. The steps are as follows:
Gait Cycle: Human walking is cyclic. We put forward our left or
right foot first and then put the right or left foot. This event occurs
repeatedly. This observation is the motivation behind looking for
cyclic pattern in the accelerometer data. While analyzing the
accelerometer data we found out that the part of data between two
consecutive peaks repeats. It has been found out that output of any
of the three axes is more erratic than the combined output of all
the three axes. For this reason we shall use the combined signal to
extract the representative gait cycle. Like [15] we have found out
that among various methods of combination the following works
best:
4. iPhone SENSORS
iPhone has several inbuilt sensors which can be utilized to record
the owner’s characteristics. iPhone has a tri-axial accelerometer, a
built-in GPS module, an ambient light sensor, a camera and a
microphone which can be used to record gait, location tracks,
ambient brightness of light, face and fingerprint and voice of the
owner respectively. Below we discuss the sensors used in our
developed application.
Accelerometer: The iPhone has three accelerometers, one
working along each of the three primary axes of the device. The
x-axis measures along the short side, the y-axis measures along
the long side and the z-axis is a line perpendicular to the iPhone
through its center [22]. Values are given in terms of “g”, where 1g
is the force of gravity.
Ri = ‫ି݊݅ݏ‬ଵ (
௓೔
ට௑೔మ ା ௒೔ మ ା ௓೔ మ
), i = 1… k
Where Xi, Yi, Zi and Ri are vertical, forward-backward, sideway
and combined acceleration at the observation number i; k is the
number of recorded observations in the signal. The combined gait
signal is the angle between the resultant signal
(݅. ݁. , ඥܺ௜ ଶ + ܻ௜ ଶ + ܼ௜ ଶ ) and the sideway axis (i.e., Z). An
example of acceleration signals in three directions and the
combined gait signal is shown in Figure 3.
Figure 2. Axes on iPhone
Three types of filtering: none, low-pass filtering and high-pass
filtering are available for the accelerometer data. High-pass
filtering helps to get rid of the gravitational effects and find out
the instantaneous movement of the device. As gait is represented
as the collection of the instantaneous movements of the device,
we have used high-pass filtering on the accelerometer’s raw data.
(a) x-Axis Data
X, Y, and Z values are linear acceleration values. They point to
whichever way gravity is operating. Together, X, Y and Z form a
3-D acceleration vector that indicates the direction of gravity [22].
GPS Module: A-GPS (Assisted GPS) on iPhone 3GS finds the
closest satellite to quickly identify the phone’s location [23]. In
case, a satellite is not within a clear line of sight iPhone finds the
location via Wi-Fi. If Wi-Fi is not available, iPhone uses cell
tower triangulation to find out its location.
(b) y-Axis Data
Other Sensors: iPhone has ambient light sensor, 3 Mega pixel
camera, and a microphone which can be used to find patterns
respectively in shifting ambient brightness, face and fingerprint,
and voice.
5. ePet ALGORITHMS
Though the architecture of our solution includes arbitrary number
(N) of primary biometrics or environmental metrics, in the present
(c) z-Axis Data
15
security. As Dynamic Time Warping algorithm allows
acceleration and deceleration of signals along the time dimension
[21], it is suitable for matching between different modes of
walking. FastDTW is a linear time and space implementation of
DTW which has quadratic complexity and thus suitable for
constrained devices like iPhone.
Currently we have used a Java based implementation [25] of
FastDTW and we have detected the cycles from the graph
manually.
(d) Combined Signal Data
Familiar Places: We have collected time-stamped location data
(latitude and longitude) of a person every 30 minutes during a
whole day. When a person enters into some place and stays there
for a while, the location data doesn’t change significantly. We can
find out the places a person visits by comparing consecutive
location information. If the location data have not changed
significantly the person has not moved to a different place. Thus
the places he has visited during a day can be marked. Thus a list
of places over several days can be collected and then a list of
places the owner usually visits can be made. After the training
phase, whenever the cellular phone finds itself in a place which is
absent from the list, it can increase its gait similarity requirement
as mentioned previously. If the owner is hurt in the leg and thus
cannot meet the stringent requirement but he wants the cellular
phone to trust the new place he is currently visiting, he can do so
by modifying the list after he has satisfied the cellular phone
through some higher level of security checking.
Figure 3. Single User Data
In Figure 4, the combined gait data of two different persons have
been plotted.
(a) Person A
6. FUTURE WORKS
Following are the future improvements we hope to do to our
solution:
•
We shall devise and implement an algorithm to automatically
recognize the gait cycle on iPhone.
•
We shall implement the FastDTW on iPhone.
•
We shall implement the place recognition algorithm on
iPhone.
•
We shall analyze the battery consumption of iPhone due to
the authentication thread.
•
We shall implement other biometrics like file system usage,
network usage, phone browsing history, face, voice,
fingerprint, hand, and ambient light patterns on the iPhone.
•
We hope to build a complete battery and memory efficient
authentication application that will authenticate the user
continually according to our proposed architecture.
•
We shall evaluate our algorithms in terms of false acceptance
and false rejection.
(b) Person B
Figure 4. Multiple User Data
In the above three figures (Figure 4(a) and (b)) gait cycle of each
person has been marked. It is clear that cycles of one person’s gait
are very similar while two cycles of two different persons are
quite dissimilar.
Gait data is a time series. So, we have looked into various
matching algorithms for time series. A summary of findings and
why we have chosen a variant of Dynamic Time Warping (DTW)
algorithm called FastDTW are given below:
7. CONCLUSION
We have proposed a non-obtrusive, reliable and adaptive solution
to cellular phone security. We have proposed to utilize
physiological and behavioral biometrics along with environmental
factors to authenticate the owner of a cellular phone. We have
deferred entering any memory based password to the last level of
security which will hopefully be needed very rarely. The cellular
phone is assumed to act like a pet animal of the owner which will
adapt itself with time and get attached to the owner so much that it
will be able to distinguish whenever it is being carried by
someone other than the owner and take security measure
automatically. We have used iPhone’s accelerometer and GPS
module to use gait and location track as authentication metrics.
Figure 5. Decision Tree
In Figure 5 a decision tree [20] has been depicted. We need a
sophisticated matching of gait cycles due to the sensitive issue of
16
7. REFERENCES
[12] Liu Rong, Duan Zhiguo, Zhou Jianzhong, and Liu Ming.
2007. Identification of Individual Walking Patterns Using
Gait Acceleration. The 1st International Conference On
Bioinformatics And Biomedical Engineering, 2007. ICBBE
2007: 543-546.
[1] Jani Mäntyjärvi, Mikko Lindholm, Elena Vildjiounaite, SatuMarja Mäkelä, and Heikki Ailisto. Identifying Users Of
Portable Devices From Gait Pattern With Accelerometers.
http://www.vtt.fi/inf/julkaisut/muut/2005/ICASSP05.pdf
[2] Ioannis G. Damousis, Dimitrios Tzovaras, and Evangelos
Bekiaris. Unobtrusive Multimodal Biometric Authentication:
The HUMABIO Project Concept.
http://www.hindawi.com/journals/asp/2008/265767.abs.html
[13] Toshiki Iso, Kenichi Yamazaki. 2006. Gait analyzer based on
a cell phone with a single three-axis accelerometer. Mobile
HCI 2006: 141-144
[3] Jonas Richiardi et al. 2005. A multimodal biometric
authentication framework, October 2005, 3rd COST 275
Workshop, Hatfield, UK.
[14] Davrondzhon Gafurov, Kirsi Helkala, Torkjel Søndrol:
Biometric Gait Authentication Using Accelerometer Sensor.
JCP 1(7): 51-59 (2006)
[4] Pamela Briggs, Patrick L. Olivier. 2008. Biometric daemons:
authentication via electronic pets. CHI Extended Abstracts
2008: 2423-2432
[15] Stan Salvador, Philip Chan: Toward accurate dynamic time
warping in linear time and space. Intell. Data Anal. 11(5):
561-580 (2007)
[5] Ning Wang, Eliathamby Ambikairajah, Branko G. Celler,
and Nigel H. Lovell. 2008. Accelerometry Based
Classification Of Gait Patterns Using Empirical Mode
Decomposition. IEEE Conference on Acoustics, Speech, and
Signal Processing, 2008. ICASSP 2008. Pages 617-620.
[16] Sausan Yazji, Xi Chen, Robert P. Dick and
Peter Scheuermann. 2009. Implicit User Re-authentication
for Mobile Devices. SpringerLink. ISBN: 9783642028298.
Pages: 325-339.
[17] Sevasti Karatzouni, Nathan Clarke: Keystroke Analysis for
Thumb-based Keyboards on Mobile Devices. SEC 2007:
253-263.
[6] Arun Ross and Anil K. Jain. 2004. Multimodal Biometrics:
An Overview. Proc. of 12th European Signal Processing
Conference (EUSIPCO), (Vienna, Austria), pp. 1221-1224,
September 2004.
[18] http://www.techchee.com/2007/07/27/additional-securityfor-cell-phone-iris-scanning/
[7] John Krumm. 2007. Inference Attacks on Location Tracks.
Fifth International Conference on Pervasive Computing
(Pervasive 2007), May 13-16, 2007, Toronto, Ontario,
Canada.
[19] Joanna Rokita, Adam Krzyzak, Ching Y. Suen: Cell Phones
Personal Authentication Systems Using Multimodal
Biometrics. ICIAR 2008: 1013-1022
[20] Dennis Shasha and Yunyue Zhu. High Performance
Discovery in Time Series: Techniques and Case Studies.
Springer, pp. 87-99. NewYork, USA, 2004.
[8] Davrondzhon Gafurov and Einar Snekkenes. Gait
Recognition Using Wearable Motion Recording Sensors.
EURASIP Journal on Advances in Signal Processing, 2009,
Special Issue on "Recent Advances in Biometric Systems: A
Signal Processing Perspective".
[21] Nong Ye ed. The Handbook of Data Mining. Lawrence
Erlbaum Associates, Inc., Publishers. New Jersey, USA,
2003.
[9] Choon-Young Lee and Ju-Jang Lee. Estimation Of Walking
Behavior Using Accelerometers In Gait Rehabilitation.
http://koasas.kaist.ac.kr/bitstream/10203/8430/1/IntJ_061.pdf
[22] http://www.wavefrontlabs.com/Wavefront_Labs/Accelerome
ter_Data.html
[23] http://www.apple.com/iphone/iphone-3gs/hightechnology.html
[10] Jiayang Liu, Zhen Wang, Lin Zhong, Jehan Wickramasuriya,
and Venu Vasudevan. uWave: Accelerometer-based
personalized gesture recognition and its applications. in IEEE
Int. Conf. Pervasive Computing and Communication
(PerCom), March 2009.
[24] http://java-ml.sourceforge.net/
[25] http://en.wikipedia.org/wiki/Mobile_banking.
[11] Marc Bächlin, Johannes Schumm, Daniel Roggen, and
Gerhard Tröster. 2009. Quantifying Gait Similarity: User
Authentication and Real-World Challenge. ICB 2009: 10401049.
17