W
E
R
C ‐
(WERC)
S
A
T
Ryan Hahn and Joshua Thomas
Introduc on Assessing Human Stress Via Voice Analysis EEG for Emo on Recogni on
WirelessEnabledRemoteCo‐Presence(WERC)isasystem
thatwillallowasinglelifecoachorpersonalassistanttore‐
motelymonitorandcommunicatewithmultipleparticipants
whohavecognitiveorbehavioralchallenges.Therecipientof
theseassistiveserviceswearsalanyardsuspendedsmartphone
thatisinterfacedwithvariousbio‐sensorstoassessthestress
levelofaparticipant(seeFig.1).Thiswillallowforvoluntary
orautomaticinterventionswhentheparticipantbecomes
stressedincertainsituations.Thegoalisfortherecipientof
theseservicestohavemoreindependenceandsenseofpur‐
poseintheirjobsettingand/orlivingenvironments.
VoiceAnalysisistheWERCwareprocessofusingarti icialintelligentmeanstoanalyzethe
slightcharacteristicchangesofaperson’svoiceinordertodetectnegativestress.Inoursys‐
tem,aJavaprogramwaswrittentoaccept ive‐secondsound ilespreviouslyharvestedfrom
thesmartphone’sinternalmicrophone.Datamanipulationandcalculationtechniques,
suchasaFastFourierTrans‐
form(FFT),normalization,and
averagefrequencybandenergy
collection,areappliedtothesig‐
naldatatopresentitinadiffer‐
entform.A32‐inputArti icial
NeuralNetwork(ANN)isthen
createdandself‐trainedbythe
Backpropagationalgorithm(see
Fig.2).Oncetrained,theANNis
abletopredicthumanstressin
realtime,theresultofwhichis
senttothesmartphoneasa1
(stressed)or0(notstressed).
Electroencephalography(EEG)isamethodofmeasuringbrainwavesignalfrequen‐
cies,usuallyusingaheadsetwithbio‐potentialsensorscapableofdetectingtheelectrical
impulsesofthebrain.TheWERCteamhasbeentestinga5‐channelheadsetbyEmotiv
alongwithprocessingmetricsandadisplaytoolthatrevealslevelsofemotionalstates
overtime.Wehopetolearnhowtorecognizehumanstressdirectlyfromthesebrainwave
patterns,soastohaveknowledgeofthestressstatusofaWERCwareconsumerduring
non‐verbalmoments,otherwiseundetectablebyourVoiceAnalysissubsystem.
Figure2.Anexampleofa32‐inputBackpropagationANN
Results
90%Accuracy!!
TotesttheANN,60sound
ilesweresplitintopreliminary
data(40 iles)andexperimental
data(20 iles).TheANNwas
trainedonthepreliminary iles
andthentestedinrealtimeon
theexperimental iles.Asthe
trainingerroroftheANNwasin‐
creasedbysmallincrements,the
accuracyoftheANNinpredicting
stressremainedat80‐90%until
a2.5‐3%error,whereitde‐
creasedto~50‐75%(seeFig.3).
Overtraining(trainingerrortoo
Figure3.AGraphdepictingtheaccuracyoftheANN
low)causesdecreasedeffective‐
withregardtoitstrainingerror
nesswithexperimentalsamples.
Figure1.TheDesignoftheWERCsystem
Thebio‐sensorelementoftheWERCsystemisintendedtoen‐
ableanautomaticinterventionalcalltothelifecoachtriggeredby
thedetectionofelevatedhumanstress.GalvanicSkinResponse
(GSR)wasusedinpastyears,butduetoitsinabilitytodistinguish
negativefrompositivestressindicators,otheroptionswereex‐
plored.TheWERCteamhasidenti iedandexploredtwoother
negativestressmeasuringtechnologiestobeusedinconjunction
withGSR:VoiceAnalysis(VA)andElectroencephalography(EEG).
Clients
Further Informa on TheclientfortheWirelessEnabledRemote
Co‐presence(WERC)projectisCurtByers.
Curtisalifecoachwhocurrentlyworkswith
individualswhohavemental,social,orcognitivechallenges.He
isanintegralpartoftheWERCprojectandhasbeenproviding
essentialfeedbackregardingtechnologicalrecommendations,
theprojecthistoryandscope,anddetailsaboutthetarget
community.
MoreinformationonArti icialNeuralNetworks:
DataMining:PracticalMachineLearningToolsandTechniquesbyIanH.Witten&EibeFrank
http://www.natureofcode.com/book/chapter‐10‐neural‐networks/
MoreinformationonEEG:
https://emotiv.zendesk.com/hc/en‐us/articles/208378593‐Frequency‐Bands‐what‐are‐they‐
and‐how‐do‐I‐access‐them‐
Figure4.Agraphdepictingemotionlevelsderivedfrombrainwaves
Figure4aboveshowswhattheoutputofananalyzedEEGsignallookslike.Weuseda
programdevelopedbyEmotivtoqualitativelytesttheabilitiesoftheheadsetinemotion
detection.Theapplicationprocessestherawfrequencyoutputwaves,Theta,Gamma,Beta,
andAlpha,incombinationwiththeamplitudeofeachtodetermineanemotionstate.Since
Emotivcurrentlyretainstheproprietaryrightstotheseemotionstatealgorithms,the
WERCteamwillneedtolearnhowtodevelopalgorithmsforprocessingthesesignalsso
astodetecthumanstress.SomeconcernswiththecurrentEEGheadsetincludephysical
discomfortafterlongperiodsofuseandrelativelyshortbatterylife.WeexpectEmotivto
addresstheseconcernswithimprovementsintheirheadsetproductgoingforward.
Conclusions and Future Work OurresultssuggestthatVoiceAnalysishaspromiseasaviablemeansofdetermining
negativestressinrealtime.MoreresearchandtestingneedstobedonewithEEGtoseeif
itiseffective.Thesetwosensortechnologieswilllikelybeusedinsomecombination
alongwithGalvanicSkinResponse(GSR)soastovalidatethedetectionofnegativehu‐
manstressmorereliablyandunderdifferentcircumstances.Thenextstepsinvolving
thesebio‐sensorsaretowriteanyextracodethatisnecessarytodeterminestressfrom
thebio‐metricdataandalsotoestablishapure,reliabledatatransferconnectionwiththe
AndroidapplicationonthesmartphoneviaUSBorBluetooth.
Acknowledgements
TheWERCteamwouldliketoacknowledgeDr.HaroldUnderwood,Dr.GeneChase,Dr.
RandallFish,andMr.CurtByersfortheirsupport,leadership,andadvice.