1
Rapid Prototype Development for Studying Human Activity
A. Fevgas1, P. Tsompanopoulou1,2, and S. Lalis1,2
1
2
Computer & Communication Eng. Dept., University of Thessaly, Volos, Greece
Centre for Research and Technology – Thessaly (CE.RE.TE.TH.), Volos, Greece
Abstract— In recent years there has been an extremely
growing interest in the study of human motion. A large
amount of scientific research projects deal with problems like
monitoring human motion, gesture and posture recognition,
fall detection etc. Wearable computers and electronic textiles
have been successfully used for the study of human physiology,
rehabilitation and ergonomics. We present a platform and a
methodology for rapid prototype development of e-textile
applications for human activity monitoring.
Keywords— Gait analysis, motion analysis, wearables, electronic textiles, prototype construction
I. INTRODUCTION
The study of human activity is one of the most popular
research areas in the recent years, with several applications
in healthcare, sports and electronic games. Many research
efforts have dealt with fall detection, gesture, posture and
activity recognition. The progress in wearable computers
and electronic textiles has significantly contributed to this
growth, as their main asset is that are pervasive during the
activity.
Wearable sensor networks (WSNs) comprise devices attached to human body, which acquire, process and/or
transmit sensor data to a host device such as a PDA, mobile
phone or wristwatch. A wireless sensor infrastructure for
healthcare applications is presented in [1]. It introduces
three types of devices: fixed devices, mobile personal devices and mobile sensor nodes which communicate using
802.15.4/ZigBee protocols. [2] introduces a wearable sensor
network infrastructure, where the sensor nodes (called Cookies) have the size of a small coin and communicate with
the outside world through a linux based host (called Muffin)
that acts as the gateway. Cinnamon, a high level programming interface, can be used to access the sensor data.
E-textiles are a particularly challenging area of research,
aiming to make computing technology truly wearable by
integrating it into fabrics. Related activities involve, among
others, creation of woven sensors [3], stretchable electronics
[4], development of computation models [5], and construction of prototype platforms [6][7]. For example, e-TAGs,
originally, presented in [6], are small computational devices
This study was supported by the Innovation Pole, Center for Research and
Technology Thessaly (CE.RE.TE.TH).
which can be attached to fabric textiles. They comprise four
different node types (master, microphone, led, input) that
are based on PIC microcontrollers. Lilypad [7] is a construction kit for building electronic textiles that includes a
central processing node and several sensors as well as actuator boards. Lilypad was originally based on conductive
laser-cut fabric PCB. It can be programmed in C via Arduino open source environment.
Wearable accelerometers have been successfully used for
capturing physical activity. [8] introduced a real time human movement classifier using a 3-axis accelerometer unit
placed in the waist. The system processes the acceleration
data and discriminates between activity and rest (with accuracy 100%), detects falls (accuracy 95.6%), and classifies
walking (accuracy 83.3%). Accelerometers and gyroscopes
have been used in [9] to distinguish between certain levels
of expertise and examine the quality of executing movements in martial arts. A large amount of work also has been
done on fall detection. An initial effort to design a reliable
fall detector is presented in [10]. A fall detector embedded
in a wrist watch is presented in [11]. It detects forward falls
by accuracy of 100%, while in backwards and sidewards the
success is reduced to 58% and 45%, respectively. In [12]
two ±50g, 2-axis accelerometers are placed orthogonally
behind the user’s ear lobe. The rationale behind placing
sensors on the head is based on the assumption that high
acceleration values in this area are associated by abnormal
situations like falls.
II. SUPPORT FOR RAPID PROTOTYPING
The design and development of wearable applications for
human activity monitoring involve decisions about the sensors and their placement, data processing, communications,
power consumption etc. Often, design adoptions (e.g., sensor position) are based on empirical criteria [9][12] and/or
experimental data [10]. Also, prototypes are used to verify
the desired properties and revise the design as needed.
However, building prototypes to verify design goals may
end to increased cost and production time. In an attempt to
address this problem, we have designed and implemented a
platform for the rapid development of e-textile and smart
clothes applications for human activity monitoring. The
platform consists of two elements: (i) a toolbox for analyz-
2
ing position and acceleration data and (ii) a hardware infrastructure for fast construction of wearable prototypes. The
toolbox can be used to process motion data in order to provide an accurate estimation of acceleration in different body
points enabling the user to choose the points that accelerometers could be placed on. Moreover, it could be used in
the design and verification of data processing algorithms
prior to any prototype manufacturing. In the other hand, the
hardware infrastructure provides the required components
for building prototypes of smart clothes in a straightforward
fashion. It also enables the recording of data acquired by
such prototypes, which in turn can be fed back to the motion
analysis toolbox.
III. HARDWARE INFRASTRUCTURE
The proposed rapid prototype development environment
provides a hardware infrastructure for constructing smart
clothes. Factors like cost, usage, power consumption, communication, programming and scaling, have been considered in the platform design.
A processing node is considered with several sensors and
actuators connected directly on it, but placed independently
of it. The node is equipped with wireless communication for
both software update and interaction with other systems. It
acquires data from sensors, processes them and if it's necessary activates actuator(s) and/or establishes communication
with external systems. Node's functionality can be changed
through application code update. The processing node, as
well as its peripherals (sensors and actuators), are battery
powered. More than one nodes can be incorporated into a
smart clothe for both scaling and redundancy.
A first prototype board, Fig. 1(left), has been constructed
for the evaluation of the design approach, followed by a
final, more compact, version, Fig. 1(right). It uses a
PIC18LF4550, 8-bit microcontroller from Microchip. This
microcontroller incorporates 35 I/O ports, 13 A/D channels,
2KB of data memory and 32KB of flash program memory
and several peripherals with a 10-bit AD converter and a
USART module among them. It supports a wide range of
operating frequencies from 32KHz to 48MHz. It also provides a range of features that reduce significantly the power
consumption. Another important feature is its selfprogrammability, enabling it to write in its own program
memory under software control. Wireless capability is provided using an XBee module from Digi (ex Maxstream) a
low-power, low-cost RF communication device that is build
on ZigBee/802.15.4.
Fig. 1 Two prototypes of the processing node
In the current implementation the microcontroller is
clocked at 12MHz at an operating voltage of 3.3V, consuming 4.36mA in idle state, 5.76mA when sampling an accelerometer and 57.09mA when transmitting the samples over
wireless. Power comes from a 6V, 128mAh PX28L battery
through an LM2937-3.3 regulator from National Semiconductors.
As has been mentioned above, the proposed system
should support application code update. A bootloader and a
host application have been developed for this reason. The
bootloader supports read, write, erase and execute application commands. The host application provides all required
functionality to control programming procedure through a
convenient user interface. The bootloader was developed in
C, based on the architecture described in [13] and the host
application was developed in Java.
Wearable sensor networks and e-textile platforms like eTags and Lilypad can also be used for prototyping smart
clothes. In WSNs a wireless node can be placed on any
body point where a sensor or an actuator is required by the
application. This introduces maintenance overhead of the
wireless nodes (e.g., battery replacement), moreover complicates communication and software development. Electronic textiles based on E-Tags have similar drawbacks, as
different type of nodes are used. LilyPad introduces a different model, where a single processing node is used and
connected to many sensors, actuators and power nodes.
Our proposed architecture was designed simultaneously
with Lilypad and follows a similar approach while also
adopting techniques used in body area networks.
The hardware infrastructure is designed to support flexible, efficient and robust data acquisition and processing
while being able to scale to large numbers of sensors and
actuators. Also, care is taken to support connectivity with
external systems, such as personal devices.
3
IV. MODA
Motion Data Analysis (MoDA) is a toolbox implemented
in MATLAB and it is designed to process and display motion data (i.e., position or acceleration) of human activities.
Its graphical user interface allows the user to process data
with ease, using existing functionalities or adding new user
defined functions specific for each particular case.
Data about the position of specific points of the human
body are collected in labs equipped with motion capture
systems. Special markers are placed on the body of a human
model, and special cameras are used to record their 3D
position during the experiment. The resulting position data
is post-processed using interpolation and differentiation
methods to provide the acceleration of the corresponding
body parts in time. Experiments on interpolation using the
Spline Toolbox of MATLAB were done to figure out the
most appropriate method for computing acceleration data.
The smoothed cubic splines with a user-defined tolerance
proved to give the best results, thus was adopted for MoDA.
However, the choice of the tolerance greatly affects the
quality/noise of the data and choosing the right tolerance
requires considerable experience.
As an alternative option, acceleration data can be directly
produced using high accuracy accelerometers stitched on
the human model, which can then be imported to MoDA for
further processing.
MoDA enables the user to visualize, smooth, observe,
compute special quantities and properties, and study different body motions (i.e., walking, running falling etc). Fig. 2
depicts a MoDA observation window showing the graphs of
3D location data (in blue) and corresponding acceleration
data (in red) for a marker placed on the waist of a person
during a fall. The right part of the window is used to zoom
into the regions denoted by the black rectangles in the left
graphs. The user can study the data for several markers/accelerometers in order to decide which body points
provide the most characteristic information for a certain
motion of interest, and construct a prototype that features
sensors on exactly these points (for further experimentation). Also, MoDA provides the ability to study the behavior of a particular body point by processing data from different motion cases. Moreover, location and acceleration
data can be post-processed and combined with each other
using readily available or user-defined functions.
V. PLATFORM EVALUATION
The platform was evaluated by building a smart jacket
for fall detection, a popular research subject in movement
analysis, with many applications in healthcare, sports, etc.
Fig. 2 Observation window picturing data of a fall
MoDA was utilized to study falls by using position data
obtained from [14] and downloaded from [15]. The acceleration amplitude on each axis (Fig. 2) was studied along
with the Euclidean norm for different body spots in order to
locate suitable points for placing acceleration sensors. It
was figured out that waist acceleration values do not differ
significantly for various fall types. Considering the observation that all recovery attempts are initiated with upper extremity movement, an accelerometer on the waist was used
for fall detection and an accelerometer on each wrist to
identify subject’s condition (responding or not) after a fall.
MoDA was also used for designing and testing the fall
detection algorithm. Position data were used, along with
acceleration values gathered by an acceleration logger attached to the waist. The collected acceleration data include
the gravitational vector of acceleration, which was also
exploited to compute the postural orientation of the subject.
An algorithm was designed based on a waist acceleration
threshold in combination with the torso orientation. The
severity of the fall is inferred using wrist acceleration values
and an alarm is raised if the subject is not responding at all
or recovery fails.
To build our fall detector (Fig. 3), a processing node and
three accelerometers were attached on a sports jacket. The
accelerometers used were 3-axis ADXL330 from Analog
Devices with range ±3g, integrated to the DE-ACCM3D
sensor board of Dimension Engineering which features
integrated op amp buffers for direct connection to the analog inputs of the microcontroller. To enhance user comfort
the processing node was placed in the spine above waist.
The sensors were attached to the wrists and waist, according
to the findings of fall analysis carried out by MoDA. The
bootloader and the programming environment were used to
program the smart jacket with the fall detection application.
Notably, the application can be uploaded on the jacket over
wireless, simplifying the testing of different versions.
4
evaluation of a smart jacket for fall detection, confirm the
usefulness of the platform. Our future work aims to enhance
MoDA’s functionality, as well as to develop a high level
programming interface for the processing node.
REFERENCES
1.
2.
Fig. 3 Smart jacket
3.
A set of experiments were accomplished to verify the
functionality of the smart jacket. The evaluation scenarios
comprised by intentional falls and daily living activities
(ADL) (as listed in Table I). The latter were included to
check for false positives. The tests were performed by three
healthy subjects of ages 24 to 32 and all activities were
repeated three times for each subject. None of the ADL was
mistakenly recognized as a fall, while all falls were detected
successfully except the backward fall wherein torso remained on upright stance.
Table 1
Activity
Daily Living
Falls
Evaluation scenarios
4.
5.
6.
7.
8.
Description
From standing position sit down to a chair
From seated position stand up
From standing position lying down to a bed
From lying position stand up
Walking regularly for 15m
Walking fast for 15m
Walking up to a 15-step staircase
Walking down to a 15-step staircase
Fall forward from a standing position, ending lying
Fall forward while walking, ending lying
Fall lateral from a standing position, ending lying
Fall backward from a standing position, ending lying
Fall backward from a standing position ending with
torso upright
9.
10.
11.
12.
13.
14.
VI.
CONCLUSIONS
We presented a platform for rapid prototyping of e-textile
and smart clothes applications that monitor human activities. The platform comprises a toolbox named MoDA for
studying human motion data and a hardware infrastructure
for building prototypes. MoDA provides studies for motion
problems, by processing available data, prior to hardware
involvement. First results, based on the development and
15.
Arriola A, Brebels S, Valderas D et al. (2008) A wireless sensor
network infrastructure for personal monitoring, 5th International
Workshop on Wearable Micro and Nanosystems for Personalized
Health, Valencia, Spain, 2008.
Hanaoka K, Takagi A, Nakajima T (2006) A Software Infrastructure
for Wearable Sensor Networks, IEEE Proc. vol. 0, International
Workshop on Real-Time Computing Systems and Applications
(RTCSA’06), Sydney, Australia, 2006, pp. 27-35.
Paradiso R, Loriga G, Taccini N (2005) A wearable healthcare system
based on knitted integrated sensors, IEEE Trans Inf Tech Biomed,
9:337-344.
Loher T, Manessis D, Heinrich R et al. (2007) Stretchable electronic
systems, IEEE Proc. Conference on Electronics Packaging Technology, Singapore, 2006, pp. 271 – 276.
Marculescu D, Marculescu R, Khosla P (2002) Challenges and opportunities in electronic textiles modeling and optimization ACM, Conference on Design Automation, New Orleans, USA, pp. 175-180.
Lehn D, Neely C, Schoonover K, Jones M et al. (2004) e-TAGS: etextile attached gadgets, Proc. Conference on Communication Networks and Distributed Systems Modeling and Simulation, San Diego,
California, USA.
Buechley L, Eisenberg M (2008) The LilyPad Arduino: toward
wearable engineering for everyone. IEEE Perv. Comp. 7(2):12-15.
Karantonis D, Narayanan M, Mathie M, et al. (2006) Implementation
of a real-time human movement classifier using a triaxial accelerometer for ambulatory monitoring. IEEE Trans Inf. Tech. Biomed. 10(1):
156-167.
Heinz E, Kunze K, Gruber M et al. (2006) Using wearable sensors for
real-time recognition tasks in games of martial arts - an initial experiment, IEEE Symposium on Computational Intelligence and
Games pp. 98-102.
Doughty K, Lewis R, McIntosh A (2000) The design of a practical
and reliable fall detector for community and institutional telecare. J
Telemed Telecare, 6: 50-54
Degen T, Jaeckel H, Rufer M et al. (2003) SPEEDY: a fall detector in
a wrist watch, 7th IEEE International Symposium on Wearable Computers, NY, USA, pp. 184 - 187.
Lindemann U, Hock A, Stuber M et al. (2005) Evaluation of a fall
detector based on accelerometers: a pilot study. Springer, Med Bio
Eng Comp, 43(5): 548-551
Fosler R, Richey R (2002) A FLASH bootloader for PIC16 and
PIC18 devices at http://www.microchip.com
Laboratory for Human Movement Analysis, CERETETH,
http://www.inhuper.cereteth.gr/laboratories/biomechanics?set_langua
ge=en
Motion Capture Database, CMU at http://mocap.cs.cmu.edu/
Author: Athanasios Fevgas
Institute:
Street:
City:
Country:
Email:
Computer & Communication Eng Dept, Univ of Thessaly
Glavani 37
Volos
Greece
[email protected]