Wireless Enabled Remote Co-presence (WERCware)
Application Development and Shutoff Solution
Joseph Coshun and Matthew Bohn
Introduction
WERCware System Layout
Wireless Enabled Remote Co-presence (WERC) is a system
that will eventually allow a single life coach or personal assistant
to remotely monitor and communicate with multiple people who
have cognitive or behavioral challenges. This assistive
communication system is intended to reduce the dependency of the
participant on the life-coach through the use of a smartphone with
associated technology. During the work day, the participant carries
a dedicated smartphone interfaced with various bio-sensors that
allows for voluntary or automatic interventions when needed.
The WERCware system implements an android smartphone, multiple biometric sensors
detecting human stress level, and an automatic shutoff solution.
The current vision of the WERCware system involves two
main functions: automatically contacting the coach if the
participant’s stress becomes too high, and automatically disabling
the connection if the user enters a private area. To achieve this, the
system must detect human stress level and signal beacons that
define private areas.
The WERC app will also be able to detect if the user has
entered a private/confidential area. In this case, the app will
automatically disable audio and video monitoring if the participant
forgets to do so manually. Private locations will be set prior to
initial use by placing a Bluetooth device in that area.
The application (app) on the smartphone will act as a central hub for communication
between the separate parts of the WERCware system. It will record audio files of the user
talking, and send these for Voice Analysis. The EEG, GSR, and VA will feed their signals to
the app, which will relay any relevant information to the life coach, and initiate a video call if
necessary. The application will also communicate with the shutoff solution; when in an area
defined by the Estimote™ beacons, it will shut down audio and video recording. Recording
will not restart until the participant has left the defined area, and has pressed a button on the
phone to restart it.
Shutoff Solution
When the user of the WERCware system enters a
private area such as a restroom or other restricted
setting, it is necessary to disable all audio and video
streams from the phone to prevent any accidental
breach of confidentiality. By placing a Bluetooth
beacon in the private area, the WERC app detects the
Figure 2. Assembly view of the Estimote™
signal and automatically disables data transmission. Bluetooth
beacon, used to detect private locations.
The Estimote™ beacon’s range is adjustable, so it
can be customized for the WERCware system to accomplish this purpose.
The smartphone used for the WERC system will run a mobile
application that combines all of the functions of the system. This
allows all biometric sensors to communicate with the phone, and
for the system to perform the tasks of initiating a video call to the
service provider when a condition has been met.
Detecting stress in an individual can be done using complex
algorithms to analyze the various signals from external sensors. A
combination of Voice Analysis (VA), brainwave monitoring
(EEG), and Galvanic Skin Response (GSR) will be able to
characterize human stress. The WERC app will then process the
signals from each sensor in real time, and if a critical threshold has
been exceeded, a Skype video call will be initiated.
Mobile Application Design
Figure 1. Cur rent design of the WERCware system. VA, EEG, and GSR sensors help detect human
stress, and the automatic shutoff solution prevents audio/video transmission when in private areas.
Most recently, the project team created a standalone Android app capable of showing the
notification “WERCware disabled” upon entering the beacon’s region. The app controls the
phone’s bluetooth radio and actively monitors for the presence of a beacon. The code from
this app will be added to the main WERCware app for final integration.
Three possible methods will be used to detect human stress:
Voice Analysis (VA) is a process of analyzing the frequencies in a per son’s voice to detect
stress. An artificial neural network (ANN) detects the higher frequencies, and associates them with
an elevated stress level. The smartphone’s microphone will collect audio to sample the user’s voice.
Electroencephalography (EEG) is the measurement of brainwaves, which are capable of
indicating stress and emotional states. An Emotiv Insight will be used to measure brainwaves. The
signal is transmitted via Bluetooth to the smartphone.
Galvanic Skin Response (GSR) is a measure of the conductivity of skin. The A ffectiva QSensor consists of two electrodes that contact the skin. The signal is transmitted via Bluetooth.
Conclusions and Future Work
The main WERCware app currently samples audio for the Voice Analysis subsystem and stops audio/video
transmission when triggered by the Automatic Shutoff subsystem. Bluetooth beacons have been successfully
connected to the smartphone and the phone can detect when a beacon is in range. While some problems exist
in the process of detecting a beacon, further testing will reveal which settings are necessary to adjust.
Additional app development will improve the functions of both the overall app and the Shutoff subsystem.
Clients
The client for the Wireless Enabled Remote
Co-presence (WERCware) project is Curt Byers.
Curt is a life coach who currently works with
individuals who have mental, social, or cognitive challenges. He is an
integral part of the WERCware project and has been providing essential
feedback regarding technological recommendations, the project history
and scope, and details about the target community.
Additional Information
The Automatic Shutoff Solution uses Bluetooth Low Energy (BLE) beacons. More information at estimote.com.
Android Studio is the programming environment used to for coding the main WERC app and the shutoff system.
Learn more about the Android API at developer.android.com/sdk/index.html.
Please see the second WERCware poster for more information on the stress detection systems.
Acknowledgements
The team would like to acknowledge Dr. Harold Underwood, Dr. Gene Chase, Dr. Nancy Patrick, and Dr.
Randall Fish, as well as Mr. Curt Byers for their support, leadership, and advice.
Photos obtained from: estimote.com/press-kit, emotive.com/media, and empatica.com/e4-wristband. Logos
used with the permission of Messiah College, the Department of Engineering, and The Collaboratory.