Hand and Facial Gestures in Conversational Interaction
OxfordHandbooksOnline
HandandFacialGesturesinConversationalInteraction
JanetBavelas,JenniferGerwing,andSaraHealing
TheOxfordHandbookofLanguageandSocialPsychology(Forthcoming)
EditedbyThomasM.Holtgraves
OnlinePublicationDate: Mar
2014
Subject: Psychology,PersonalityandSocialPsychology
DOI: 10.1093/oxfordhb/9780199838639.013.008
AbstractandKeywords
Conversationalhandandfacialgesturesareanintegralpartoflanguageuseinface-to-facedialogue.Extensive
researchshowsthatconversationalhandgesturesaretightlysynchronizedwithwordstodemonstrateanything
thatcanberepresented(directlyormetaphorically)assize,shape,position,oractionandthattheyarehighly
sensitivetotheimmediatecommunicativecontext.Althoughresearchonconversational(nonemotional)facial
gesturesismuchmorelimited,they,too,arepreciselytimedwithwordsandcontexttodemonstrateanythingthat
canberepresentedbyafacialconfiguration—whetherinthepast,present,orfutureandevenhypotheticalor
metaphorical.Bothhandandfacialgesturescanalsofunctionascollateralcommunication(meta-communication).
Thischapterincludestheoretical,methodological,andtechnicalrequirementsforstudyingthesegesturesin
conversationalinteraction.
Keywords:Conversationalhandgestures,conversationalfacialgestures,languageuse,dialogue,conversationalinteraction,collateral
communication
Inface-to-facedialogue,interlocutorsspontaneouslycombinewords,prosody,handgestures,andfacialgestures
toproducerapidlychangingintegratedmessages(Bavelas&Chovil,2000,2006).Thischapterpresentsa
frameworkforstudyingtheseintegratedmessagesassocialcommunicationandthenanoverviewofwhat
experimentalresearchisdiscoveringabouthowthevisibleelementsofcommunicationcontributetolanguageuse
inconversationalinteraction.
LookingCloselyatConversationalHandandFacialGestures
Toappreciatethedynamic,fleetingqualityofconversationalhandandfacialgestures,itishelpfultosee(rather
thantoreaddescriptionsof)asmanyexamplesaspossible.Thischapterusessequentialframeshotsfromour
researchvideos,aformatthatmayhelpthereaderimaginehowtheseactionsactuallylooked(andmayleadto
noticingthemmoreoftenineverydaylife).Inhisadviceonhowtogetthemostoutofexamplespresentedinprint
form,McNeill(1985,p.352)urgedreaderstoactouttheexamplesinordertoexperiencethesinglecoordinated
actionofmotor–speechsynchronythattheexamplesareintendedtoshow.
Figure1.presentsthreeframesina6.75-secondvideoexcerpt(fromBavelas,Gerwing,&Healing,2014).The
speaker’staskwastowatchseveralscenesfromtheanimatedmovieShrek2andthenretellthemtoan
addressee.ShehadjustfinisheddescribinganattackonShrekbythecatcharacter(PussinBoots).Inframe1,
shebegantodescribethenextscene,inwhichShrekpickedupthedefeatedcatbythebackoftheneck,lifted
himclosetohisface,andthecatstartedbeggingforhislife.Byframe2,shehadraisedherhandupinfrontofher
ownface,pinchingherfirsttwofingersandthumbasifliftingandthensuspendingthecat,presentinganimageof
Shrekholdingthecatinfrontofhisface.Thishandgesturedemonstratedinformationthatwaseitherleft
ambiguousinherwords(e.g.,theshapeofShrek’shand)orwasmissingfromherwords(e.g.,thatShrek
Page 1 of 22
PRINTED FROM OXFORD HANDBOOKS ONLINE (www.oxfordhandbooks.com). (c) Oxford University Press, 2014. All Rights
Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a title in Oxford
Handbooks Online for personal use (for details see Privacy Policy).
Subscriber: Oxford University Press - Master Gratis Access; date: 05 August 2014
Hand and Facial Gestures in Conversational Interaction
suspendedthecatintheairatfacelevel).
Clicktoviewlarger
Figure1 .WhiledescribingascenefromthemovieShrek2,thespeakermadethehandgestureinframe2
andthefacialgesturesinframes2and3.Thethreeframeshotscovera6.75-secondperiod.(Thefaceof
theaddressee,whowassittingacrossatablefromthespeaker,isinsetattheupperright.)
Thespeaker’sfacewasalsoactive.Inframe2,whilesaying“neck,”shedemonstratedShrek’sslightlygloating
expressionasheheldthecatathismercy.Inframe3,shebegantoquotethecat(“OhI’msosorry!”)while
wrinklingherforeheadandpursingherlips,creatingaworriedandpleadinglook.Eachofthesefacialgestures
portrayedoneofthecharactersandillustratedhernarrativefortheaddressee.Likeherhandgestures,each
addedrelevantdetailsthatherwordsdidnotconvey(e.g.,Shrek’sexpressionasheheldthecat,thenthecat’s
apprehension).
SomeDefinitionsandAssumptions
ThecloseanalysisofthethreeframesinFigure1.illustratesthedefiningcharacteristicsofconversationalhand
andfacialgestures.Eachgesturecomplementedthespeaker’swordsandwastimedpreciselywiththerelevant
words.Bothhandandfacialgesturesoccuraspartofaspontaneousstreamofspeech,andtheyaretightly
synchronizedinbothmeaningandtimingwiththeaccompanyingwords.Thespeaker’shandgestureinframe2
beganandendedexactlywiththewordsitwasrelatedto.Thefaceishighlymobile,sospeakerscanmakefacial
gesturesevenmorequicklythanhandgestures—attherateofasinglewordorevensyllable.Infact,Figure1.
includesonlytwoofthespeaker’sfacialgesturesintheexcerpt;thereweretwomore(onebeforeandoneafter
frame3).Altogether,in4seconds,shemadefourdifferentfacialgesturesrelatedtowhatshewassaying.
Thesynchronybetweenvisibleandaudibleelementsofcommunicationresultsinefficienciesthatcouldnotbe
matchedbywordsalone.Evenifwordscouldconveyexactlythesameinformation,thespeakerwouldbelimited
topresentingitinasequential,linearmanner(e.g.,“Shrekpickshimupbythenapeofhisneckwithhisthumb
andtwofingersandholdshimupateyelevelinfrontofhisface,andlooksathiminsortofagloatingway.”)
Theadditionalwordsneededtoconveyinformationthathadbeenconveyedbythespeaker’sgestureswould
requireamuchlongerdescriptionthantheoriginalintegratedmessage.
Thischaptertreatshandandfacialgestures—butnotallnonverbalbehaviors—aspartoflanguageuseinface-tofacedialogue.Thereareseveraltypesofnonverbalbehaviorsthatarenotpartoflanguageuse,includingthose
thatareinvoluntaryorreflexive(e.g.,blinking),instrumental(e.g.,turningadoorkey),staticpostures(e.g.,sitting
withcrossedarms),orbehaviorsthatarejustaslikelytooccurwhennooneelseispresent(e.g.,squintingin
brightlight).Inaddition,thehandandfacialgesturesdiscussedindetailinthischapterarethosethatoccurwithin
thecontextofconversation,notthoseoccurringinnonspeakingcontextssuchashandsignals(e.g.,thehitchhikingsign)orstillphotos.
TheRoleofContextinMeaning
InFigure1.themeaningofeachofthespeaker’shandandfacialgesturesdependedonthesetofnestedcontexts
inwhichtheyoccurred.Figure2.presentsonewaytovisualizethesecontexts.Thesettingwasauniversity
psychologyexperimentforclasscredit.Theinterlocutorswereunacquaintedstudents.Theirtaskwasforthe
speakertotelltheaddresseethescenesfromShrek2.Theirdialoguesofarconsistedoftheirconversationupto
thispoint,whichculminatedinaparticularmicrosocialmomentinthedialogue(Bavelas,2007),forexample,
demonstratingShrekpickingupthecat.Inadifferentsetofnestedcontexts,ahandandfacialgesturecombination
similartotheoneinframe2wouldhaveanentirelydifferentmeaning:imaginethemeaningofthesegesturesifshe
hadbeeninapubtalkingwithfriendsabouthowmuchfunitwouldbetodropawaterballoononherboss.
Page 2 of 22
PRINTED FROM OXFORD HANDBOOKS ONLINE (www.oxfordhandbooks.com). (c) Oxford University Press, 2014. All Rights
Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a title in Oxford
Handbooks Online for personal use (for details see Privacy Policy).
Subscriber: Oxford University Press - Master Gratis Access; date: 05 August 2014
Hand and Facial Gestures in Conversational Interaction
Clicktoviewlarger
Figure2 .Imageofthenestedcontextsthatsuccessivelydeterminethemeaningofwords,aswellashand
andfacialgesturesinface-to-facedialogue.
Noticethatthesamelayersofcontextalsodeterminethemeaningofanywordorphraseinadialogue.For
example,inthisparticularcontext,“thecat”unambiguouslymeantthePussinBootscharacterratherthanany
otherfeline.Ameaning-basedanalysisofhandgestures,facialgestures,andwordsmustalwaystakeintoaccount
thesecontextuallevels—whichiswhateveryonedoeseveryday,ineverydialogue.
HandandFacialGesturesasDemonstrations
Wehaveproposed(Bavelas,Gerwing,Sutton,&Prevost,2008;Bavelasetal.,2014)thatbothconversationalhand
andfacialgesturesfitintoalargerviewoflanguage,communication,andthegeneraltheoryofsignsinitiatedby
CharlesPeirce(e.g.,Buchler,1940).Peircedistinguishedbetweensymbols(e.g.,words),indices(e.g.,aweather
vane),andicons(e.g.,apicture).ClarkandGerrig(1990;Clark,1996,chapter6)putthesethreecategorical
distinctionsintoasocialcontextbyemphasizinghowtheyfunctionineverydaylanguageuse:withsymbols,a
speakercandescribesomethingtoanaddresseebyusingwordsintheirconventional,arbitrarilyestablished
meanings.Usingindices,aspeakercanalsoindicatesomethingfortheaddresseebypointingatit,either
physicallyorwithademonstrativepronoun(e.g.,“Iwantthatone”).Usingicons,aspeakercandemonstrate
somethingbypresentingavisualorauditoryversionofit.Demonstrationsincludedirectquotations(Clark&Gerrig,
1990),metaphors,conversationalfacialgestures,andconversationalhandgestures(Bavelasetal.,2014).
Demonstrationsshareseveraldefiningfeatures.Ratherthanhavingstandardizedmeanings,they“workby
enablingotherstoexperiencewhatitisliketoperceivethethingsdepicted”(Clark&Gerrig,1990,p.765).They
dosobyresemblingtheirreferentinsomeway.Althoughademonstrationresemblesitsreferent,itisalsoa
selectivetransformationoftheliteraloractualpropertiesofthereferent;itselectssomefeaturesandleavesout
others.Thegesturaldemonstrationinframe2ofFigure1.presentedthespeaker’sversionofthesceneshehad
seen.HerhandresembledShrek’shandholdingthecat,butitwasnotanexactreplication.Sheselected(and
emphasized)theimageofShrekholdingthecatbetweenhisfingersinfrontofhisface(butdidnotincludeShrek
bouncinghishandslightly,asshowninthemovie).Inthesameframe,herfaceresembledhowShreklooked,but
sheselectedamildlygloatingandvictoriousversionandleftouttheangerthatwasclearlypresentinhisfacein
theactualmoviescene.
AFunctional,CommunicativeApproach
Thischapterconcernsthecommunicativefunctionsofhandandfacialgesturesinsocialinteraction.First,the
focusisonfunctionratherthantype.Thatis,wewillbeasking“Whatisthisgesturedoinginthisdialogue?”rather
than“Whatcategorydoesthisgesturegoin?”Afunctionalapproachleadstotermssuchasdemonstrated,
portrayed,presented,complemented,andmarked,allofwhichdirectattentiontothefunctionofthegestureinits
Page 3 of 22
PRINTED FROM OXFORD HANDBOOKS ONLINE (www.oxfordhandbooks.com). (c) Oxford University Press, 2014. All Rights
Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a title in Oxford
Handbooks Online for personal use (for details see Privacy Policy).
Subscriber: Oxford University Press - Master Gratis Access; date: 05 August 2014
Hand and Facial Gestures in Conversational Interaction
microsocialcontext.Althoughhandgesturescanbeclassifiedasiconicsorillustrators,suchcategorization
abstractsthemfromtheirinteractionalcontext.Second,ourfocusoncommunicativefunctionsteersawayfrom
questionssuchas“Whatdoesthisfacialgesturetellusaboutwhattheindividualisfeeling?”Inferencesaboutthe
cognitiveoremotionalprocessesassociatedwithahandorfacialgesturearedistinctfromexaminingwhatitis
doingintheinteraction.Functionsarenotexclusive:actionsthatarecommunicativemayhavecognitiveor
emotionalfunctionsaswell.Althoughthereisnologicalreasontoprivilegeonefunctionoveranother(muchlessto
settheminopposition),theydorequirequitedifferentresearchmethods.Inanycase,contemporaryresearchhas
movedpasttheadversarialdebateaboutwhethergesturescommunicatetothemoreinterestingquestionofhow
theydoso,whichopensupentirelynewpossibilitiesforresearchandtheorizing.
MethodologicalIssues
Asimpliedinthepreviousparagraph,experimentsonthecommunicativefunctionsofconversationalhandand
facialgesturesrequirearesearchdesigninwhichoneconditionisaface-to-facedialogue.Thenextsections
explicatethisrequirementandcriticallyexaminesomeconventionalbarrierstomeetingit.
ConversationalInteractionasFace-to-faceDialogue
Conversationalhandandfacialgesturesservetheircommunicativefunctionswithinface-to-facedialogues.Clark
(1996,pp.9–10)specifiedthedefiningcharacteristicsofface-to-facedialogue,whichstresstwokeyprinciples:
first,aface-to-facedialogueisunmediated,whichmeansthattheinterlocutorsareinthesamephysical
environment,abletoseeandheareachother,andabletoproduceandreceiveatonceandsimultaneously.
Second,theinterlocutorscanfreelyinteractandcollaboratewitheachother,momentbymoment.InClark’sterms,
theyareactingasthemselves(e.g.,notasaconfederate),determiningwhatactionstotakewhen(e.g.,withouta
script),andactingextemporaneously,inrealtime.Thesefeaturesprovideessentialcriteriaforresearchdesigns
thatwillyielddatarelevanttoface-to-facedialogue.
TrueDialoguesVersusQuasi-Dialogues
Traditionally,experimentalsocialpsychologistshaveshownamethodologicalpreferenceforwhatBavelasand
Healing(2013)calledquasi-dialogues,inwhichtheparticipantinteractswithsomeonewhoisactingunder
instructionsthatconstrainhisorheractions(e.g.,aconfederate,theexperimenter,oranotherparticipant).These
interactionsdonotfitClark’s(1996)definitionoftruedialoguesbecauseoneoftheparticipantsisnotactingas
him-orherself,isnotdetermininghisorherownactions,andisnotabletoextemporize.Thissectionexploresthe
reasonsunderlyingthechoiceofquasi-dialogues.(SeealsoKuhlen&Brennan,2012.)
First,manyscholarshavepointedoutthatsocialpsychologyhasremainedlargelythestudyoftheindividual
ratherthanthedyad(e.g.,Bavelas,2005,2007;Clark,1985;Danziger,1990;Sears,1951;Thibaut&Kelley,
1959).AsDanzigerpointedout,thelinebetweenthepersonandtheenvironmenthashistoricallybeendrawn
tightlyaroundtheindividual,sothattheinterlocutorinadialogueis“outside”thisline—justanotherpartofthe
environment.However,whentheinterestisinhowthespeaker’sactionsfunctionindialogue,thentheunitofstudy
mustincludethedialogueitself.
Evensocialpsychologistswithaninterestincommunicationduringsocialinteractionoftenusequasi-dialogues,
perhapsbecauseofanerroneousapplicationoftheprincipleofreductionism.Luria(2004,p.537)described
reductionismastheassumptionthat“thebasicgoalofscienceistoreducecomplexphenomenatoseparate
simpleparts,andthatsuchreductionprovidessignificantexplanationsofphenomena.”Bythisprinciple,atrue
dialogueisacomplexphenomenathatcanbebetterstudiedbyreducingittooneindividualinacontrolled
interaction.However,Luriawentontoquestionthisassumption:
Althoughthephilosophyofreductionismwasacceptedasageneralprincipleinthenaturalsciencesand
psychology,therearegroundstosupposeitmaybefalse.Tostudyaphenomenon,oranevent,andto
explainit,onehastopreserveallitsbasicfeatures:onemustbeabletodescribetheirrulesandtheir
mechanismswithoutthelossofany[characteristicsofthephenomenon].Itcaneasilybeseenthat
reductionismmayverysoonconflictwiththisgoal.(p.537)
Page 4 of 22
PRINTED FROM OXFORD HANDBOOKS ONLINE (www.oxfordhandbooks.com). (c) Oxford University Press, 2014. All Rights
Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a title in Oxford
Handbooks Online for personal use (for details see Privacy Policy).
Subscriber: Oxford University Press - Master Gratis Access; date: 05 August 2014
Hand and Facial Gestures in Conversational Interaction
Intheircriticalexaminationofreductionism,Reber,Allen,andReber(2009,p.663)pointedoutthattheproblemis
notwithreductionismasascientificprinciplepersebutwithitsovergeneralizationtoalltopicsinallfields.
Reducingacomplexphenomenontoitssmallestpossibleunitsmakessenseifthesmallerunitsarethetopicof
interest.Butstudyingthesmallerunitswillnotbeinformativeifamorecomplexphenomenonofinterestislostin
theprocess.Forexample,ifoneisinterestedinlearningaboutsalt(sodiumchloride),thenstudyingtheproperties
ofsodium(ametalsoreactivethatitalmostexplodesinwater)andchlorine(apoisongas)wouldnotbevery
enlightening.Ifthephenomenonofinterestissocialinteractionindialogue,thenstudyingindividualsissimilarly
unenlightening.
Forexperimentalresearchers,reductionismalsounderliesasecondjustificationfortheuseofquasi-dialogues,
namely,theassumptionthatthecomplexityofarealdialogueprecludesexperimentalcontrol.Thefearisthatif
oneinterlocutoractsspontaneouslyandtheotherisalsofreetoreactspontaneously,experimentalcontrolwillbe
lostandtheensuingvariabilitywilloverwhelmanydifferencesbetweenconditions.However,face-to-facedialogue
isinherentlywellorganized.Ineverydaylife,evenstrangersregularlyaccomplishcomplextaskstogetherin
dialogue(e.g.,givingdirections,discussingpurchases,exchangingtechnicalinformation,makingadiagnosis,or
conductinganinterview).Thenumerousexperimentsreviewedinthischapterdemonstratethatallowingtwo
individualstorespondextemporaneouslytoeachotherwithinanassignedtaskdoesnotleadtounanalyzable
data.Farfrombeingchaotic,dialoguesareintrinsicallyorderly;thoseofuswhomicroanalyzedialogueare
constantlyawedbyitsprecisionandorderliness.
SomeTechnicalIssues
Asapracticalmatter,observingtheprecisionofface-to-facedialoguerequiresdigitizedvideowithboth
interlocutorsonscreenatalltimes(e.g.,usingsplit-screen)plussoftwareforframe-by-frameanalysisand
annotation(e.g.,ELAN,http://tla.mpi.nl/tools/tla-tools/elan/;Wittenburg,Brugman,Russel,Klassman,&Sloetjes,
2006).Usingcomprehensiveoperationaldefinitions,analystscancapturethedetailsofdialoguewithhigh
interanalystagreement.(Gullberg’s,2010,informativeoutlineofmethodologicalconcernsingestureresearchin
secondlanguageacquisitionisrelevanttogestureresearchmorewidely,especiallypp.86–91.)
ConversationalHandGestures
Imagineyouarequietlyobservingtwopeoplehavingaconversationinarestaurant.Youcan’theartheirspeech,
butyoucanseethattheymakeavarietyofhandmovements.Notallofthesemovementsareconversationalhand
gestures.First,someareinstrumentalactionsinvolvingobjects(e.g.,cuttingtheirfoodwithaknifeandfork)or
theirownbody(e.g.,pushinghairbehindanear).Theseactionsareoftencalledobject-andself-oriented
adaptors,andwhentheyservepurelyinstrumentalfunctions,theyhavenocommunicativemeaning.Second,you
mayseeonepersonpointtoacorneroftherestaurant,promptingtheotherdinertolookinthatdirectionaswell.
Thesepointing(indicatingordeictic)gesturesfunctiontoestablishajointfocusofattention.Third,onepersonat
thetablecatchestheserver’seyeandholdsonepalmupwhilepretendingtowriteonitwiththeotherhand.Ifyou
arefamiliarwiththeculturalnormsofthegesturer,youunderstandthismovementasarequestforthebill.Ifthe
servernodsandthencomestothetablewiththebill,thegesturewaseffective.Thesehandsignals,whichare
usuallycalledemblems,havestereotypicformsandculturallyspecifiedmeaningssotheycanfunctionas
messagesoutsideofconversation.Asidefrominstrumentalactions,pointing,andemblems,youseebothdiners
movingtheirhandsastheytalk.Theseimprovisedmovementsseemtomeansomething,butyouwon’tknowtheir
meaningwithouthearingthediners’wordsbecausethesearelikelytobeconversational(orco-speech)hand
gestures,asdefinedintheopeningparagraphsofthischapter.(Forathoroughreviewofthehistoryandvarietyof
gestures,seeKendon,2004.)
Conversationalhandgestureshaveatleastthreephases(Kendon,1980).Mostgesturesbeginwithapreparation
phase,liftingormovingthehandsintoposition.Themeaningfuldepictiveactionthatfollowsistheactualgesture
stroke.Afterthestroke,thereisoftensomeretraction,asthehandsmovebacktoarestingpositionorpreparefor
thenextgesture.Immediatelybeforeorafterthestroke,theremayalsobeabriefhold,inwhichthehandspause
inposition.Thesesequencesoccursmoothly,inamatterofsecondsorless,andtheyaretightlysynchronized
withthestreamofspeech.
Page 5 of 22
PRINTED FROM OXFORD HANDBOOKS ONLINE (www.oxfordhandbooks.com). (c) Oxford University Press, 2014. All Rights
Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a title in Oxford
Handbooks Online for personal use (for details see Privacy Policy).
Subscriber: Oxford University Press - Master Gratis Access; date: 05 August 2014
Hand and Facial Gestures in Conversational Interaction
Thefollowingsectionsfocus,first,onwhatkindsofinformationthegesturestrokecancontributetoadialogueand,
second,onhowsocialfactorsaffecttheminadialogue.Thestudiesreferredtointhissectionwereexperimentsin
whichatleastoneconditionfulfilledallofthecriteriaforaface-to-facedialogue.
WhatCanHandGesturesDemonstrate?
Handgesturesaresuitedtodepictingsomekindsofinformation(e.g.,shape,size,ordirection)andnotothers
(e.g.,color,flavor,ornames).Still,aswillbeseeninthissection,thevarietyofpossibilitiesissurprising.Mostof
theexamplesarefromspontaneousconversationalhandgesturesmadeduringlaboratoryexperimentsusing
dialogictasksdesignedtoelicitgestures(e.g.,retellinganarrative,describinganobject,orexplainingtoan
addresseehowtodosomething).Theyarehighlyvariedbecauseeachwasademonstrationspecifictoa
particularmicromomentinadialogue.Recallthatdemonstrationsdonothavestandardizedconventional
meanings.Instead,theyresembletheirreferentandrequiretheaddresseetorecognizethisparticularhandaction,
inthiscontext,asaselectiveversionofthereferent.
HandActions
Gesturesarewellsuitedtodemonstratinghandactionsthatarerealorhypothetical;literalormetaphorical;donein
thepast,present,orfuture;andrepresentingone’sownoranyone’shands.InanexperimentbyGerwingand
Bavelas(2004),speakersfirstusedasetofobjectsandlaterdemonstratedtheiractionstoanaddressee(without
theobject).Oneoftheseobjectswasawoventube-liketoythattrappedtheusers’indexfingerswhentheytriedto
pullthemout.Whentellingtheiraddresseeaboutit,speakersusuallygesturedtheiractions,oftenbyholdingtheir
indexfingersinastraight,horizontallinewiththetipstouching,sotheaddresseecouldimagineatubeholdingthe
fingersinthisposition.Sometimestheypulledtheirfingersbackandforthtodemonstratehowthefingerswere(or
couldhavebeen)trappedinthetube.Anotherobjectwasapropeller-liketoythatcouldbelaunchedbyspinning
theuprightstickbetweenverticalpalms.Speakerslatergesturedhowtheyhadlaunchedthetoybymovingone
verticalpalmrapidlypasttheother.InastudybyFuruyama(2000),speakerstaughtaddresseeshowtheyshould
foldanorigamifigure—withoutanypaper.Theteachersusedgesturestodemonstratemakingthevariousfolds.
Thus,theaddresseehadtoseethehandsasthoughtheyweremanipulatingapieceofpaperthatwasnotactually
there.
Speakersalsodemonstratehandactionsotherthantheirown,aswhenthespeakerinFigure1.usedherhandto
representShrek’shand.Theaddresseehadtoseethespeaker’shandasifitwereShrek’s.Inanotherexperiment
(Bavelasetal.,2008),speakershadtodescribeapictureofawomanholdingafaninfrontofherchest.Speakers
oftenincludedagesturaldemonstrationofthefan,asifholdingafanthemselves.
Finally,gesturesofhandactionscanbemetaphorical.InanexperimentbyRowbotham,Holler,Lloyd,and
Wearden(2011),interviewersaskedparticipantstodescribearecentpainexperience.Althoughpainisan
intangible,subjectiveexperience,theparticipantsoftenusedgesturestodemonstratevariousaspectsoftheir
pain.Onepersonsaid,“Itfeelslikethey’re[sic]justsattherewithlikeahammer,hittingme,that’showitfeels.”
Duringtheitalicizedwords,thespeakergesturedasifholdingahammerandhittingherselfrepeatedlyonher
temple(p.8).Herdemonstrationrequiredtheaddresseenotonlyto“see”thisactionbutalsotoappreciatethat
theheadachefeltasifthehammerwerepoundinghertemple.
SpatialFeatures
Handsarenotlimitedtoportrayinghandactions.Speakerscanusetheirgesturestodemonstratephysicalfeatures
ofanobjectsuchasitsshape,size,ororientation.Forexample,somespeakersintheGerwingandBavelas(2004)
experimentusedtheirhandstodepictthepropellertoyitself.Figure3.presentsonesuchpairofgestures:inframe
1,preciselyasthespeakersaid“arod,”shebegantobringheruprightindexfingerstraightdown,drawinga
verticallinefortherod.Asshesaid“thentwo”(inframe2),shedrewahorizontallinetorepresentthepropellers
atthetopoftherod.Inframe3,sheheldhergestureattheendofthehorizontallinewhileshefinishedsaying
“propellers.”
Page 6 of 22
PRINTED FROM OXFORD HANDBOOKS ONLINE (www.oxfordhandbooks.com). (c) Oxford University Press, 2014. All Rights
Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a title in Oxford
Handbooks Online for personal use (for details see Privacy Policy).
Subscriber: Oxford University Press - Master Gratis Access; date: 05 August 2014
Hand and Facial Gestures in Conversational Interaction
Clicktoviewlarger
Figure3 .Whiledescribingapropellertoytoheraddressee,thespeakermadethreerelatedhandgestures.
Thethreeframeshotscovera1.4-secondperiod.(Theaddresseewassittingdirectlyacrossfromthe
speakerandfilmedinsplitscreen,indicatedherebyadottedline.)
Inotherexperiments,speakershaveusedgesturestodemonstratetheshapeofknottedpipes,dome-shaped
roofs,orbridges(Holler&Stevens,2007;seetheirfigures1–3,pp.17–19),theevolvingshapeofanorigamifigure
(Furuyama,2000),andtheshapeandsizeofdistinctivefeaturesofanunusualdress(Bavelasetal.,2008;see
theirfigures2–4,pp.504–507).
Spatialgesturescanalsorepresentvirtualspaces,thatis,whenthereferentdoesnotactuallyexist.Bavelas,
Gerwing,Allison,andSutton(2011)askedparticipantstodesignafloorplantogetherwithoutpencilandpaper.
They“drew”theirplansonthetablewiththeirgestures,demonstratingthelocation,size,andshapeofvirtual
roomsthatdidnotactuallyexist.
Asinglegesturecanbedensewithinformation,thatis,demonstratingmorethanonefeatureofthereferent
(Gerwing&Allison,2011).Inseveraloftheexamplesdescribedinthissection,gesturesdepictingshapealso
conveyedaccurateinformationaboutthereferent’ssize(e.g.,Holler&Stevens,2007;Bavelasetal.,2008)or
otherfeaturessuchaslocation(Gerwing&Allison,2009,seetheirFigure1.p.313)ororientation.
Motion
Handscanmoveinalmostanydirectionandarewellsuitedtodemonstratingaspectsofmotion.Inanexperiment
byParrill(2008),participantsretoldacartooninwhichacathadswallowedabowlingball,whichthenbeganrolling
downahill,transportingthehelplesscatalongwithit.Speakersoftenabstractedandselectedaspectsofthis
motiontodemonstratewiththeirgestures.Forexample,Parrill’sfiguresillustrategesturesthatdepictedthe
directionthecatrolled(movingthehanddiagonallydownward;herFigure1.p.287),therollingmannerofthe
movement(tracingacircularmotioninoneplace;herFigure4.p.289),aswellasbothdirectionandmanner
simultaneously(tracingacircularmotionmovingdownwards;herFigure5.p.290).InÖzyürek’s(2002)
experiment,participantsusedgesturestoshowthedirectionofmotionascharacterswentintooroutofbuildings
oracrossastreet(Özyürek,appendix,p.703).Gesturesdepictingmotioncanalsobemetaphorical;forexample,
representingthelesseningofpainbymovingthehanddownward(Rowbothametal.,2011,table4).
Clicktoviewlarger
Figure4 .Thespeaker’sandaddressee’sfacesinframe3showedtheirpersonalreactionsasthespeaker
wastellingaboutascenefromShrek2.Thethreeframeshotscovera6.2-secondperiod.(Thefaceofthe
addressee,whowassittingacrossatablefromthespeaker,isinsetattheupperright.)
Clicktoviewlarger
Figure5 .Theaddressee(insetattheupperleft)wassittingdirectlyacrossfromthespeaker,listeningtothe
speaker’sclose-callstory.Inframe2,hisfacialgestureconveyedapersonalreactiontoasuddendangerin
thestory.Thethreeframeshotscover2seconds.
RelativePositions
Page 7 of 22
PRINTED FROM OXFORD HANDBOOKS ONLINE (www.oxfordhandbooks.com). (c) Oxford University Press, 2014. All Rights
Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a title in Oxford
Handbooks Online for personal use (for details see Privacy Policy).
Subscriber: Oxford University Press - Master Gratis Access; date: 05 August 2014
Hand and Facial Gestures in Conversational Interaction
Somegesturesdemonstrateboththereferentanditsrelationshiptosomethingelse,suchasanothergesture,the
gesturer’sbody,oranearbyobject.Indiscussionsaboutfloorplans,GerwingandAllison(2009)foundtwo-handed
gesturesthatdemonstratedtherelativepositionsoftwoormorerooms.Forexample,oneparticipantgesturedand
heldthelocationoftheentrancewithherrightindexfingerwhileshowingtheproposedlocationofthekitchenwith
herlefthand.Shethenheldherlefthandinpositionwhilemovingherrighthandovertotheproposedlocationof
thelivingroom(Gerwing&Allison,2009,theirFigure1.p.313).
Therestofaspeaker’sbodyprovidesanadditionalresource:speakerscanusetherelationshipbetweentheir
hand(s)andbodytodemonstraterelativepositions.InFigure1.ofthischapter,thespeakerusedtherelationship
betweenhergestureandherfacetodepictwhereShreksuspendedthecatwhilelisteningtohispleading.In
anotherexperiment(Bavelasetal.,2008),speakersdemonstratedthedistinctivefeaturesofadress.Menaswell
aswomenoftenplacedtheirgesturesinrelationtotheirownbody.Someoutlinedthebodiceofthedressasa
deep“V”overtheirownchest(Bavelasetal.,2008,theirFigure4.p.507)orshowedtheunusuallywideshapeof
theskirtbyextendingtheirhandsoutfromtheirownwaist(Gerwing&Allison,2011,theirFigure2.p.311).In
descriptionsoftheirpainexperiences,Rowbothametal.’s(2011)participantsalsoplacedtheirgesturesinrelation
totheirbody;forexample,oneparticipantputherpalmsoverherlowerstomachwhileleaningforwardtoshow
howshepositionedherselfasareactiontothepain(Rowbothametal.,2011,table4,p.17).Gesturescanalso
buildonthelocationofpreviousgestures,asifthepreviousgesturelefta“residue”behindthatcouldbeusedby
latergestures.Forexample,inFigure3.ofthischapter,noticethattheparticipantgesturedthepropellersdirectly
overwhereshehadpreviouslyplacedthetopoftherod.HealingandGerwing(2012)analyzedthegesturesof
speakerswhodescribedapictureofamaze-likeroute.Theyfoundthatspeakersoftenlinkedcontiguoussections
oftheroutebygesturingonesection,pausing,andthenbeginningthenextgesturefromthesameposition.
GesturesandWords
Thepreviousexamplesshowthatgesturesareparticularlysuitedformany—butnotall—kindsofinformation;for
example,althoughtheycandemonstratesomething,theycannotnameit.Wordsandgesturesareusefulfor
communicatingdifferentkindsofinformation,andBavelasandChovil(2000,2006)proposedthattheywork
togethertoformintegrated(butnotnecessarilyredundant)messages.Asemanticfeatureapproach(e.g.,Beattie
&Shovelton,1999)cananalyzehowinformationisdistributedbetweenwordsandgestures,andityieldsreliable
quantitativeevidenceofhowtheirintegrationworks.Oneapplicationofthisapproachexaminedhowdyadsused
gesturesandwordswhendesigningfloorplans(Gerwing&Allison,2009).Gesturesweresignificantlymorelikelyto
contributespatialinformation(i.e.,size,shape,andrelativelocationoftherooms)whereaswordscontributed
categoricalinformation(e.g.,thenameofaroom).Onespeakersaid“there’sakitchen”whilepointingtoaspace
adjacenttowhereshehadlocatedtheentrance.Thewordsnamedtheroom,andthegesturesspecifiedits
location.Tounderstandthespeaker’smessage,theaddresseehadtointegratespeechandgesture.Afollow-up
studyondressdescriptions(Gerwing&Allison,2011)reportedasimilarpatternofintegration:inface-to-face
dialogues,speakers’gesturesweresignificantlymorelikelythantheirwordstoconveyspatialinformationsuchas
thesizeandshapeoftheskirt.Whendescribingtheirpainexperiences,participantsinRowbothometal.(2011)
weremorelikelytoputinformationaboutthelocationandsizeoftheirpainintheirgesturesratherthanintheir
words,whileputtinginformationaboutthecause,theeffect,andtheirawarenessofthepainsignificantlymore
oftenintheirwords.
ConversationalFactorsthatAffectGestures
Thephysicalcapacitiesofhandstodemonstratereferentsarebynomeanstheonlydeterminantsofwhatgestures
actuallydoinadialogue.Theformofagestureisalsodeterminedbyitsfunctioninthedialogueattheprecise
momentitoccurs.Mostoftheexperimentscitedintheprecedingsectionswillreappearherebecausetheirprimary
purposehadbeentoshowhowfactorsinthesocialinteractioncanaffecthowandwhentheinterlocutorsuse
gestures.
MutualVisibility
Afrequentexperimentalmanipulationingestureresearchconsistsofvaryingwhetherornotthespeakerand
addresseearemutuallyvisible(e.g.,speakingface-to-facevs.throughapartition).SinceCohenandHarrison’s
Page 8 of 22
PRINTED FROM OXFORD HANDBOOKS ONLINE (www.oxfordhandbooks.com). (c) Oxford University Press, 2014. All Rights
Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a title in Oxford
Handbooks Online for personal use (for details see Privacy Policy).
Subscriber: Oxford University Press - Master Gratis Access; date: 05 August 2014
Hand and Facial Gestures in Conversational Interaction
initialexperiment(1973),researchershavehypothesizedthat,iftherateofgesturingdecreaseswhenthe
addresseeisnotvisible,thengesturesserveacommunicativefunction;ifnot,thentheyserveacognitivefunction
(e.g.,helpingthespeakerformulateutterances).
However,thereareseveralreasonstodoubttheutilityofthisdesignforsuchabroadhypothesis.First,theresults
ofnumerousexperimentsontheeffectofmutualvisibilityonoverallgestureratehavebeenmixed,withhalffinding
thatmutualvisibilityleadstoasignificantlyhigherrate(Alibali,Heath,&Myers,2001;Cohen,1977;Cohen&
Harrison,1973;Emmorey&Casey,2001;Krauss,Dushay,Chen,&Rauscher,1995;Mol,Krahmer,Maes,&Swerts
2009a,2009b)andtheotherhalffindingnosignificantdifferenceinoverallrate(Bavelasetal.,1992,2008,2014;
DeRuiter,Bangerter,&Dings,2012;Holler,Tutton,&Wilkin,2011;Pine,Burney,&Fletcher,2010;Rimé,1982).
Twoofthelatterstudiesfoundbothresults,dependingonthekindofgesturestudied(Bavelasetal.,1992;De
Ruiteretal.,2012).Thesecondproblemisthatalloftheexperimentsreportingasignificantlyhigherrateofgesture
inthevisibilityconditionwerequasi-dialogues,whereastheothergroupwerealltruedialogues.Clearly,theresults
oftheformerdonotgeneralizetothelatter.(SeeBavelas&Healing2013,foradiscussionofpossiblereasons.)
Finally,severalexperimentshaveshownthatoverallratemeasuresdonotcapturetheeffectsofmutualvisibility
ongestureuse:speakersmakequalitativelydifferentgestureswhenaddresseescanseethemthanwhenthey
cannot;seeBavelasandHealing(2013,pp.76-79,andTable5).Forexample,mutualvisibilityaffectswhetheror
notgesturesconveyessentialinformation.Whentheiraddresseewillseetheirgestures,speakersaresignificantly
morelikelytoconveyinformationinthemthatdoesnotappearinwords.Whentheiraddresseewillnotseetheir
gestures,speakers’gesturesarelesslikelytoaddanyadditionalinformation;thatis,theytendtoberedundant
withthewords(e.g.,Bavelasetal.,2008;deRuiteretal.,2012).Bavelasetal.(2008)alsofoundthatspeakers’
gesturesinface-to-faceconversationsweresignificantlylargerthanthoseonthetelephone.HealingandGerwing
(2012)foundthat,whendescribingthemaze-likeroutetoavisibleaddressee,speakersweresignificantlymore
likelytolinksequentialgesturesintoacohesivewhole,maintainingtheirrelativepositions.Whentheiraddressee
couldnotseethem,gesturesforsequentialfeatureswerenotlinkedbutinsteadtendedtopileupinonearea.
Otherresearchhasshownthatsomespecificgesturefunctionsrequirevisibility:interlocutorswhocouldseeeach
otherweremorelikelytomimictheformandmeaningofeachother’sgestures(Holler&Wilkin,2011).
SharedSocialSpace
Justasspeakerscanusetheirownbodyasaresourceforcontributingtogestures’meaning,theycanalsouse
thespacethattheysharewiththeiraddressee.Özyürek(2000,2002)comparedexperimentalconditionsinwhich
speakerswereseatedindifferentspatialconfigurationswiththeiraddressees.Forexample,theymightbeseated
directlyacrossfromtheaddresseeortalkingtotwoaddresseesinatriangularconfiguration(seeexamplesin
Özyürek’s,2002,Figure1.p.692,andFigure2.p.698).Whendemonstratinghowacartooncharactermovedfrom
oneplacetoanother,thespeakersdisplayedsensitivitytohowtherelativeseatingpositionsoftheiraddressees
andthemselveswouldinfluencethemeaningoftheirgestures.Forexample,theychangedthedirectionoftheir
gesturestomatchwhetherthegesturewasmovingintooroutofthesharedspace.Interlocutorscanevenshare
theirgesturalspace.Furuyama’s(2000)studyonorigamiinstructionreportedthatinstructorsandlearnersoften
collaboratedwiththeirgesturesinthesamephysicalspace,pointingtooractingonasingle“virtual”origamifigure
thatwasnotactuallyinthatspace.
CommonGround
Itiswellestablishedforwordsthatspeakersaresensitivetowhetherornottheysharecommongroundwiththeir
addressee(Haviland&Clark,1974).Forexample,theyprovidefuller,moreinformativeverbaldescriptionswhen
theyknowtheaddresseeisunfamiliarwiththeinformation.GerwingandBavelas(2004)showedasimilareffectfor
gesturesbymanipulatingcommongroundbetweenparticipants.Speakersdescribedthesameobjecttwice:once
toanaddresseewhotheyknewhadjustmanipulatedthesameobjectandoncetoanaddresseewhotheyknew
hadnot.Thegesturesthatspeakersusedtoidentifytheobjectfortheaddresseewhodidnotsharecommon
groundwerejudgedtoconveymoreinformation,bemorecomplex,orbemoreprecisethanthosethespeaker
usedtoidentifytheobjectfortheaddresseewhodidsharecommonground(e.g.,theirfigures3and4,pp.169–
170).HollerandStevens(2007)foundaneffectofcommongroundontherelationshipbetweenthegesturesand
wordsthatspeakersusedtoconveythesizeofseveraldisproportionatelylargeobjects.Theyfoundthatspeakers
whodescribedtheseobjectstoaddresseeswhohadnotseenthemusedgesturessignificantlymoreoftenthan
Page 9 of 22
PRINTED FROM OXFORD HANDBOOKS ONLINE (www.oxfordhandbooks.com). (c) Oxford University Press, 2014. All Rights
Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a title in Oxford
Handbooks Online for personal use (for details see Privacy Policy).
Subscriber: Oxford University Press - Master Gratis Access; date: 05 August 2014
Hand and Facial Gestures in Conversational Interaction
words.Furthermore,theirgesturesdepictedtheobjects’sizesmoreaccuratelythangesturesusedinthecommon
groundcondition.Conversely,whenparticipantssharedcommonground,speakersusedwordstorefertothe
objectssignificantlymoreoftenthantheyusedgestures.
Astheirdialogueunfolds,theinterlocutorsaccumulatemorecommonground.Speakerscanintroducenew
informationthat,whentheyrefertoitlater,becomesgiveninformationandpartoftheircommonground.Gerwing
andBavelas(2004)foundthatgesturesforanobjectwouldchangeastheinformationdepictedwentfromnewto
given.Forexample,thegestureswouldchangefromlargeandprecisetosmallerandvaguer.Thesechangesin
gesturesparallelwhathappenswithwords:newwordsthatbecomegivenchangefromnountopronounorfroma
distinctpronunciationtoashorterandlessdistinctpronunciation.Fowler(1988)pointedout(forwords)thatsuch
attenuationisnotsloppinessbutratherservesthecommunicativefunctionofmarkingthestatusoftheinformation
inthedialogue.Weproposethatattenuationservesthesamefunctionforgestures.
GroundingwithGestures
Theaccumulationandcoordinationofcommonground(grounding)isaninteractiveprocessduringwhich
participantsgiveeachotherevidencethattheyhaveunderstoodeachother“wellenoughforcurrentpurposes”
(Schober&Clark,1989,p.228).Severalresearchershaveexaminedtherolegesturesplayingroundingby
analyzingsequencesofimmediatelycontiguousutterancesthatinvolvedgestures.ClarkandKrych(2004)varied
visibilitytoexaminetheroleofgesturesinaddressees’responsestotheirspeaker’sinstructions.Ineachpairof
participants,thespeakerhadamodelconstructedfromLegoblocksandinstructedtheaddressee(whocouldnot
seethemodel)howtobuildit.Inthepairswhocouldseeeachother,addresseesoftenrespondedwithgestures
thatdemonstratedtheircurrentunderstandingoftheinstructions.Speakersmonitoredthesegesturalsignalsof
understanding,oftenfollowingupbymodifyingtheirinstructions,sometimesinmid-utterance.Intheirstudyofpairs
designingfloorplanstogether,Bavelasetal.(2011)focusedonresponsestothe552gesturesthathadcontributed
informationthatwasmissingfromthewords(i.e.,werenonredundant).Thevastmajorityofaddressees’responses
tothesegesturesindicatedunderstandingoftheinformation.Moreover,themajorityofspeakers’follow-up
responsesacceptedthattheinformationinthegesturehadbecomecommonground(e.g.,speakerspresented
newinformationthatpresupposedunderstandingofthepreviouslypresentedinformation).Theseandotherstudies
thatexaminedtherelationshipbetweeninterlocutors’behaviors(e.g.,Furuyama,2000;Holler&Wilkin,2011)
representnecessarynextstepsinunderstandinghowgesturesareintegratedsequentiallyintodialogue.
DialogueVersusMonologue
Theuseofconversationalhandgesturesseemstobetiedtodialogue,regardlessofvisibility.Bavelasetal.(2008,
2014)showedthatamajordeterminantoftheoverallrateofgesturingiswhetherthecontextisadialogueora
monologue.Theycreatedthreeexperimentalconditions:aface-to-facedialogue,adialogueonthetelephone,and
amonologueintoamicrophone.Thetelephonedialoguesmetthecriteriaforadialogue(summarizedearlier)
exceptthattheinterlocutorswerenotinthesameroomorabletoseeeachother.Inbothexperiments,thetwo
dialogueconditionselicitedasignificantlyhigherrateofgesturesthanthemonologuecondition,andtheserates
werenotsignificantlydifferentfromeachother.Speakinginamonologuesuppressedthegestureratetoabout
one-thirdofthetwodialogues.
GesturesasCollateralCommunication
Clark(1996,pp.241–243and255–257)proposedthatdialoguehastwoparalleltracks.Theprimarytrackisthe
topicoftheconversation;thesecondarytrackisabouttheprocessofhavingthedialogue(e.g.,“OK,I
understand”),whichismeta-communicationorcollateralcommunication.Weproposethathandgesturescanalso
functionascollateralcommunication.Forexample,whenspeakerschangetheformoftheirgesturetomarkitas
commonground,theirdepictionofthereferentistheprimarytrack,anditsattenuatedformisthesecondarytrack
(inthiscase,markingthe“given”statusofthisinformationinthedialogue).Entrainmentisanotherformof
collateralcommunication.ClarkandWilkes-Gibbs(1986)andSchoberandClark(1989)haveillustratedverbal
entrainment:whendescribingTangramfigures,interlocutorswhocouldnotseeeachotherentrainedonthesame
arbitraryverbaldescriptionsofthefigurestheywereworkingwith.Usingthesametaskwithinterlocutorswho
couldseeeachother,HollerandWilkin(2011)foundgesturalentrainment.Forexample,addresseesoften
mimickedthespeaker’sgesturetoshowthattheyhadunderstoodwhichTangramthespeakerwasreferringto
Page 10 of 22
PRINTED FROM OXFORD HANDBOOKS ONLINE (www.oxfordhandbooks.com). (c) Oxford University Press, 2014. All Rights
Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a title in Oxford
Handbooks Online for personal use (for details see Privacy Policy).
Subscriber: Oxford University Press - Master Gratis Access; date: 05 August 2014
Hand and Facial Gestures in Conversational Interaction
(e.g.,theirfigures2and3,p.145).Theirmimicrywasnotconveyinginformationaboutthetopic(i.e.,identifying
thefigure);itwasinsteadcollateralcommunication,showingmutualunderstandingofthatinformation.
Somegesturesoperateentirelyascollateralcommunication.Bavelasetal.(1992)andBavelas,Chovil,Coates,
andRoe(1995)identifiedasubsetofgestureswithinteractivefunctions.Thesegesturesdonotconvey
informationaboutthetopicoftheconversation.Instead,theyreferdirectlytotheaddressee,intwosenses:(a)
Thegestureisphysicallyorienteddirectlyattheaddresseewithnootherdepictivefeatures,and(b)thegesture’s
meaningreferstotheaddresseeandtotheprocessofdialogueitself.Examplesincludemovingthefingersor
palmstowardtheaddresseeasifmetaphoricallydeliveringinformationtotheaddresseeorflickingtheindexfinger
towardtheaddresseetociteapreviouscontribution(equivalentto“asyoujustsaid…”).(Seesketchesofsome
interactivegesturesinBavelasetal.,1992,theirFigure1.pp.474–475,andinBavelasetal.,1995,theirFigure1.
p.396.)Aseriesofexperimentsdesignedtoconfirmtheirinteractivefunctionshowedthatthesegesturesoccurred
atasignificantlyhigherrateinthefollowingconditions:(a)whenspeakersweretalkingtoanaddresseerather
thannarratingalonetoacamera(Bavelasetal.,1992),(b)whenspeakersweretalkingtoavisibleaddressee
ratherthantoonewhocouldnotseethem(Bavelasetal.,1992),and(c)wheninterlocutorswhocouldseeeach
otherwereworkingtogetherinaback-and-forthdialogueratherthancontributinginalternatingmonologues
(Bavelasetal.,1995).Twomicroanalysesfurtherconfirmedthehypothesizedfunctionsofinteractivegestures
(Bavelasetal.,1992,pp.483–486;Bavelasetal.,1995,study2).
SummaryofHandGestures
Anexaminationofthedatesofthepublicationsjustsampledrevealsthatexperimentalresearchonhandgestures
indialogueisgrowingrapidly.Inaddition,theInternationalSocietyforGestureStudieswithitsregularconferences,
thejournalGesture,andotherpublicationsinawiderangeofrefereedoutletsallsuggestthatthefieldisavital
one,withawidevarietyofresearchfociandestablishedmethods.Evenbetter,theresearchisopeningupnew
questionsabouthandgesturesinconversationalinteraction.
ConversationalFacialGestures
Let’sgobacktopeople-watchinginarestaurant.Ifyouarepayingcloseattentiontothediners’faces,thefirst
thingyouwouldprobablynoticeisthatfacesbearlittleresemblancetostillphotographs.Withtheirnumerous
independentmusclegroups,humanfacesareextraordinarilyfastandflexible.EkmanandFriesen(1978)identified
morethanfortydifferentactionunitscorrespondingtoactivityindistinctfacialmusclesormusclegroups,which
accountfortheabilityofthefacetomoveflexiblyandrapidly.Youmayalsonoticethatmostofthesemovements
seemtofitinthesamethreebroadfunctionsashandgestures.
First,allindividualswillhavefacialmovementsthatareself-orientedadaptorssuchasblinking,chewing,or
swallowing;theseactionsserveinstrumentalfunctions(e.g.,keepingtheeyesmoistoreating).Second,youmay
alsonoticeonepersondirectingafacialactiontowardsomeoneatadistance,eventhoughthetwoofthemarenot
engagedinspeakingwitheachother.Forexample,adinermaystaresteadilyacrosstheroomuntiltheserver
looksupandmakeseyecontact;thenthedinermightsmileslightlyandraisehisorhereyebrowsexpectantlyto
confirmarequestforattention.Thesefacialactionsinnonspeakingcontextsare(likehandgesturesinthesame
context)calledemblems(Ekman,1977).
Finally,whenyoufocusontwodinerswhoaretalkingwitheachother,youwillprobablyseeamuchwiderrangeof
facialactions:smiling,frowning,tiltingandmovingtheirheads,furrowingtheireyebrows,wideningtheireyes,
wrinklingtheirnoses,grimacing,andsoforth.Alloftheseactionsseemtomeansomething,butifyoucan’thear
whattheyaresaying,youwon’tknowwhatthesefacialactionsmean.Thesearelikelytobeconversational(orcospeech)facialgestures.Asdefinedandillustratedintheopeningparagraphsofthischapter,conversationalfacial
gesturesaretightlysynchronizedwithspeechinbothtimingandmeaning.Ifyou’vepreviouslyreadalotabout
facialexpressions,youmightassumethatthesearefacialexpressionsofemotions.However,somecautionisin
orderbecause,asexplainedinthenextsection,conversationalfacialgesturesarenotlikelytobefacial
expressionsofemotion.
FacialExpressionsVersusFacialGestures
Page 11 of 22
PRINTED FROM OXFORD HANDBOOKS ONLINE (www.oxfordhandbooks.com). (c) Oxford University Press, 2014. All Rights
Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a title in Oxford
Handbooks Online for personal use (for details see Privacy Policy).
Subscriber: Oxford University Press - Master Gratis Access; date: 05 August 2014
Hand and Facial Gestures in Conversational Interaction
KrautandJohnston(1979)introducedadistinctionbetweentwokindsoffacialactions:somethatexpressemotion
andothersthatdisplayinformationtoaninterlocutor.Facialexpressionsarepartofanemotional,individual
process,whereasconversationalfacialgestures1arepartofasocial,interactiveprocess.Weproposethatthis
distinctionleadstoatleastfourdifferences:first,conversationalfacialgesturesarepreciselytimedwithspeech,
sotheyappearandchangewithinseconds(asillustratedinFigure1.ofthischapter).Ekman(1997)pointedout
thatanemotionwouldrunitsowncourseandwouldnotgoonandoffwithasinglewordorphrase.
Second,mosttheoriesoffacialexpressionslimitthemtoasetofstereotypicformscorrespondingtoasmall
numberofbasicemotions(anger,happiness,disgust,etc.).Conversationalfacialgestures,ontheotherhand,are
partofwhateverthepersonissayingatthemoment,sotheyhaveinnumerableforms,noneofwhichisfixedor
stereotypic.Forexample,inFigure1.thereisnoemotionthatcorrespondstothespeaker’sdemonstrationofhow
Shrekorthecatlookedinthemovie.
Thethirddistinctionismethodologicalandaffectsthechoiceofresearchprocedures:becausefacialexpressions
areoftenseenasstereotypicconfigurationsthatcorrespondtoindividualemotions,atypicalexperimentwould
focusontheparticipant’sabilitytorecognizethesefacialexpressionsinstillphotographs.Incontrast,an
experimentonfacialgesturesindialoguerequiresvideorecordinginterlocutorsinface-to-facedialogue,then
microanalyzingtheirfacialgestures,secondbysecond,withtheaccompanyingwordsandcontext.
Fourth,facialgesturesincludemoreaspectsofthefacethandofacialexpressionsofemotion.Theanalysisof
emotionalexpressionshasbeenlimitedtoconfigurationsoffacialmuscles,suchasEkmanandFriesen’s(1978)
actionunits.However,facialgesturesoftenincorporateotheraspectsofthefacialarea,especiallyeyemovements
andheadpositions.Forexample,inFigure1.frame3,thetiltofthespeaker’sheadandtheupwarddirectionofher
gazeportrayedthecatdirectinghisworriedexpressionupatShrek.Herheadandeyepositionswerepartofher
facialgestureasawhole.
ConversationalFacialGesturesasDemonstrations
Thesedistinctionsmakeitclearthat,likehandgestures,conversationalfacialgesturesaredemonstrations.
BavelasandChovil(1997)firstproposedthatconversationalfacialgesturesmeetthecriteriaofbeingnonliteral,
selectivedepictionsofareferentthattheyresemble.InFigure1.thespeakerwasbrieflydepictingsomeselected
featuresofamoviecharacterwhowasgloatingandthenanothercharacterwhowasactingworried.Althoughher
featuresresembledgloatingorbeingworried,shewasnotliterallygloatinginframe2orworriedinframe3.Instead,
shewascreatinganintegratedandtightlysynchronizeddemonstrationconsistingofherwords,facialgestures,
andhandgestures,alltimedtogetherwithinfractionsofseconds.Theresultallowedheraddresseetoperceive
directlywhatthesecharacterslookedlikeatthispointinthemovie—oratleastthespeaker’sselectedversionofit.
Itishighlyunlikelythatanaddresseewouldmistakethesefacialgesturesforthespeaker’sownemotional
expressions.
Althoughthevastmajorityofresearchonfaceshasbeenconcernedwithemotionalexpressions,these
expressionsseemtobeinfrequentinconversationalinteractions.Fridlund,Ekman,andOster(1987,p.160)
reportedunpublisheddatabyEkmanandFriesenonnearly6,000facialactionsfrominterviewsinwhich
psychiatricpatientswithaffectivedisordersweremostlytalkingabouttheirfeelings.Eveninthiscontext,fewer
thanone-thirdofthesefacialactionswereclassifiableasemotionalexpressions.InChovil’s(1989,1991/1992)
datafromordinaryconversations,theclosestcategorytoemotionalexpressionswas“personalreactions”(see
PersonalReactionsbelow),whichwere24percentofthefacialgesturesshefound.However,eventhesewere
timedwithspeechandthereforedonotfitemotiontheory.Giventhatfacialactionsduringdialogueareprimarily
conversationalfacialgesturesratherthanfacialexpressionsofemotion,itisstrikingthatthereareliterally
thousandsofpublishedstudiesonfacialexpressionsofemotionbutonlyahandfulofstudiesonconversational
facialgestures.Thistopiciswideopenfororiginalresearch.
ThischapterreliesontheseminalfindingsofChovil(1989,1991/1992,2005;seealsoBavelas&Chovil,1997),
whichwasthefirstandstilltheonlyextensivedescriptionoffacialgesturesinspontaneousface-to-facedialogues.
Todiscoverinductivelywhatcontributionsfacialgesturescanmaketoconversation,Chovilvideotapedtwelve
dyads(fourmale,fourfemale,fourmixed)talkingaboutavarietyofassignedtopics(e.g.,aclose-call,an
interpersonalconflict).Thevideoscapturedclose-upsofbothinterlocutors’facesinsplitscreen.Usingthenested
Page 12 of 22
PRINTED FROM OXFORD HANDBOOKS ONLINE (www.oxfordhandbooks.com). (c) Oxford University Press, 2014. All Rights
Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a title in Oxford
Handbooks Online for personal use (for details see Privacy Policy).
Subscriber: Oxford University Press - Master Gratis Access; date: 05 August 2014
Hand and Facial Gestures in Conversational Interaction
contextapproachillustratedinFigure2.Chovilfocusedonthefunctionofeachfacialconfigurationatthat
particularmomentinthedialogueanddevelopedahighlyreliablesystematicanalysisoffacialgestures(e.g.,
Chovil,2005).Smileswereexcludedforpracticalreasons,becausetheyarebothfrequentandhighlyvaried
(Ekman,1985,p.150),sothatworkhasyettobedone.Evenso,sheidentifiedmorethanathousandfacial
actions:880wereconversationalfacialgestures,301wereadaptors,andonly3werenotanalyzable.Wehave
maintainedChovil’sspecificfunctions(includingapplyingthemtosomesmiles)buthaveintroducedsomeslightly
differentgroupingsandtermsandhaveillustratedthemwithexamplesfromourowndata(primarilyfromBavelaset
al.,2014).
Theoverviewherefollowsthesameformatasthegesturesection,firstillustratingwhatfacialgesturesareableto
demonstrateandthensummarizingthesmallexperimentalliteratureontheirconversationalinfluences.
WhatCanFacialGesturesDemonstrate?
Facespresentdifferentpossibilitiesandconstraintsthandohandgestures(or,indeed,words)andtherefore
demonstratedifferentkindsofreferents.Facialgesturescannotportrayobjects,spatialrelationships,motion,and
thelike,buttheycandepictwhatvirtuallyanyonelookedlikeatanyparticularmoment.Conversationsareoften
populatedwithoneself,someoneelse,orevenimaginarycharacters,whetherinthepast,present,orfuture,sothe
capacitytodepictpeople’sfacesisveryuseful.
PersonalReactions
Bothspeakersandaddresseescandemonstratehowtheyarereactinginthepresentsituation,thatis,their
evaluationoforfeelingaboutsomethingintheircurrentinteraction.Forexample,inFigure4.thespeakerwas
relatingasceneinShrek2duringwhichtheDonkeyaccidentallykickedShrekwhiletryingtohelphimrepelthe
cat’sattack.Thespeakersaid“andthenDonkeytriestohelpShrek,um,bykickinghiminthecrotch.”Inframe3,
assoonasshehadfinishedsayingtheword“crotch,”shesmiledandlaughedtodemonstratetotheaddressee
thatshethoughtthatthiswasfunny.Wedonotinferfromhersmileorthesmileoftheaddresseethattheywere
feelingtheemotionofhappinessatpreciselythatmoment.Whatwecanobserveisthattheirfacialreactions
demonstratedthatthescenewashumorous.
Figure5.(fromourarchives)showsadifferentpersonalreactionbyanaddressee.Thespeakerwastellinga
personalclose-callstoryaboutwalkingonawetlogtocrossafloodingriver.Thespeakersaid“andmyfootslips
offandIfellintotheriver.”Justashefinishedsayingtheword“river”(inframe1),theaddressee(inframe2)
madeashockedfacewithpursedlipsandsaid“Oooh.”Theaddressee’sfacedemonstratedspecificallyand
vividlyhisunderstandingofthespeaker’ssuddendangeratthispointinthestory.Thisfacialgesturedidnotmean
thathewasactuallyshockedandconcernedduringthe.7-seconditlasted;instead,itwastimedtofitthefinal,
culminatingpointofthespeaker’ssentence.
FacialPortrayals
Speakersandaddresseescanmovebeyondtheirpresentreactionsinthedialogueandinsteadportray
themselvesorothersinthepast,thefuture,orevenahypotheticalsituation.Frames2and3ofFigure1.showtwo
suchportrayalsinwhichthespeakerportrayedthefaceofShrekandthenthecat.Anotherexamplecomesfrom
Chovil’s(1989,1991/1992)data.Whendescribingaminorconflictinthepast,speakersoftenportrayedhowthey
hadreactedtotheotherperson—andhowtheotherpersonhadreactedtothem(e.g.,asindignant,angry,or
appeasing).Onespeakersaid“ButIwaslikeerrrrr!”Preciselyat“errrrr,”shetightenedhereyesandproducedan
angryface(1991/1992,p.182).Chovil(1989)alsoobservedthespecialcaseofaddressees’facialmotor
mimicry;forexample,whenthespeakertellsaboutaninjurythathappenedtohimorher,theaddressee’sfacial
portrayal(e.g.,awince)wouldbeappropriatetothesituationthespeakerisdescribing.
FacialMetaphors
BavelasandChovil(2000)proposedthatfacialgesturesvaryonacontinuumofabstraction.InFigures4and5,the
facialgesturesliterallyrepresentedthespeaker’sownface.InFigure1.however,thefacialgestureswere
somewhatmoreabstractbecausetheyrepresentedsomeoneelse’sface;thatis,theaddresseehadtointerpret
thespeaker’sfaceasifitwereShrek’sorthecat’sfacelookingthatway.Atanevenmoreabstractlevel,“facial
Page 13 of 22
PRINTED FROM OXFORD HANDBOOKS ONLINE (www.oxfordhandbooks.com). (c) Oxford University Press, 2014. All Rights
Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a title in Oxford
Handbooks Online for personal use (for details see Privacy Policy).
Subscriber: Oxford University Press - Master Gratis Access; date: 05 August 2014
Hand and Facial Gestures in Conversational Interaction
metaphorsconverttheliteralmeaningofafacialreactiontoanabstractonebyusingitinametaphor”(Bavelas&
Chovil,2000,pp.174–175).Forexample,whenanaddresseebeginstosuspectthatthespeakerisexaggerating
orteasing,hemaytilthisheadandsquint,asif—metaphorically—lookingmorecloselyatthestory.Oran
addresseewhoishearingafrienddescribeasocialblundermayshowapainedexpressioneventhoughthe
friend’s“pain”waspurelysocial,notphysical.Ultimately,facialgesturesmaybecomeevenmoreabstract,asin
manyofthosedescribedinthenextsection.
FacialGesturesasCollateralCommunication
Manyfacialgesturesfunctionascollateralcommunication(Clark,1996,pp.241–243,255–257)becausethey
demonstrateinformationaboutthestatusoftheongoingtalkratherthanthecontentofthetalk.Forexample,
speakersusevariousfacialgesturestostressspecificwordsorphrases,toexplainpausesforawordsearch,to
indicateaquestion,andtodemarcatethebeginning,continuation,orendofastory.Addresseesusebackchannel
facialgesturestodemonstratethattheyareattendingandfollowing.Chovil(1989,1991/1992)foundthatalmost
halfofthespeakers’facialgesturesinherdatadidnotconveytopicalinformationsuchasapersonalreactionora
portrayal.Instead,theymarkedsomeaspectofthediscourseitself.InsteadofusingChovil’sterms,weare
groupingthesefacialgesturesundercollateralcommunicationinordertoemphasizetheirfunctionalsimilarityto
collateralcommunicationbywordsorgestures.
Emphasizers,Underliners,andQuestionMarkers
InChovil’sdata,themostcommoncollateralcommunicationwiththefacewastheemphasizer(e.g.,Ekman,1979),
whichoccurredsynchronouslywithastressed(e.g.,prosodicallymarked)word.Theseweretypicallybrow
movements;thefacialmusclesthatmovethebrowsarerapidenoughtogoupanddownwithasinglewordor
evensyllable.Figure6.showsanemphasizerinframe2.ThespeakerwastalkingaboutascenefromShrek2in
whichShrekandDonkeyweresuddenlyambushedbyPussinBoots.Thespeakersaid“andtheyarestoppedand
ATTACKEDbythecat”;preciselyasshevocallystressedtheword“ATTACKED,”sheraisedandloweredher
eyebrows.Together,theforcefulprosodyandtheraisedeyebrowsemphasizedthekeywordinhersentence.
ChovilalsofoundwhatEkman(1979)hadcalledunderliners,whichmarkawholephraseratherthanasingleword.
BavelasandChovil(2000,p.175)laterspeculatedthattheraisedeyebrowscharacteristicofemphasizersand
underlinersareametaphorbasedonthestartlereaction,althoughthespeakerinFigure6.wasobviouslynot
demonstratingthatshewasliterallystartledbythisword.
Clicktoviewlarger
Figure6 .Inframe2,thespeaker’sraisedeyebrowsemphasizedthemainwordinhersentence
(“ATTACKED”).Thethreeframeshotscovera3.4-secondperiod.(Thefaceoftheaddressee,whowas
sittingacrossatablefromthespeaker,isinsetattheupperright.)
ThenextmostfrequentcollateralcommunicationinChovil’s(1989,1991/1992)datahadagrammaticalfunction,
thequestionmarker.Speakersfrequentlyraisedorloweredtheireyebrowswhenaskingaquestion,including
questionsmarkedonlywithrisingintonation.InFigure7.thespeakerstartedretellingthemoviebyintroducingthe
characters.WhenshecametotheDonkeycharacter,shesaid,“Youknow,Donkey?”andraisedherbrows
exactlywith“knowDon-”tomarkherbriefphraseasaquestion.Theaddresseenoddedandsmiledslightly,then
thespeakercontinued.Chovilalsofoundother,lessfrequentfacialgesturesthatservedascollateral
communication,includingonesthatmarkedthebeginning,continuation,orendingofatopic.Forexample,
immediatelyafterframe3inFigure1.thespeakersmiledtoclosethatpartofhernarrative.
Clicktoviewlarger
Page 14 of 22
PRINTED FROM OXFORD HANDBOOKS ONLINE (www.oxfordhandbooks.com). (c) Oxford University Press, 2014. All Rights
Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a title in Oxford
Handbooks Online for personal use (for details see Privacy Policy).
Subscriber: Oxford University Press - Master Gratis Access; date: 05 August 2014
Hand and Facial Gestures in Conversational Interaction
Figure7 .Thespeaker’sraisedeyebrowsmarkedherwordsinframe2asaquestion.Thethreeframeshots
covera2.3-secondperiod.(Thefaceoftheaddressee,whowassittingacrossatablefromthespeaker,is
insetattheupperright.)
ThinkingFacesandFacialShrugs
Thethinking(orremembering)faceisacollateralcommunicationindicatingthatthespeakeristhinkingaboutwhat
tosay,tryingtoremembersomething,orsearchingforaword.Typically,thespeakerbreakseyecontactbrieflyby
lookingaway(orclosingbotheyes)andmightalsosuddenlymakeeithera“blank”oran“effortful”expression.In
Figure8.thespeakerhadannouncedthatshewasabouttodescribethesceneinwhichShrekandDonkeymet
Puss-in-Boots;instead,shepaused,shookherheadslightly,andsaid(asifaskingherself)“Whathappens?”while
tiltingherheadandlookingup.Ittookonly1.82secondstomoveintoandholdthisgesture,whichendedassoon
asshebegantodescribethescene.Whetherornotthegestureactuallyhelpedthespeakertothinkorremember,
itdidservetheimportantsocialfunctionofexplaininghertemporaryspeakingpausetotheaddressee.(Similarly,
inFigure1.asthespeakerchangedfrombeingShrektobeingthecatduringthefractionofasecondbetween
frames2and3,shepausedat“like—”andmadeabriefthinkingface,lookingawaywithablankface.)
Clicktoviewlarger
Figure8 .WhilerecallingascenefromShrek2,thespeakermadeanextended(1.7-second)thinkingfacein
frames2and3.(Thefaceoftheaddressee,whowassittingacrossatablefromthespeaker,isinsetatthe
upperright.)
Anothercollateralcommunicationisthefacialshrug,which(likeashouldershrug)conveyseithernotknowingor
notcaringaboutsomethingatthatmomentinthedialogue(e.g.,havingsaidenoughonthetopic,havingsaidit
wellenough,concedingorresigningapoint).BothEkman(1985)andChovil(1989,1991/1992)observedfacial
shrugs,whichtypicallyinvolveaquickeyebrowflashandtheretractionofacornerofthemouth.Figure9.shows
thespeakershruggingwithherfacebypullingthecornerofhermouthinward.Thisfacialshrugoccurredasshe
cametotheendofasceneintheShrekmovieandpausedbetweentwopartsofherutterance.Shesaid“and
then,”madethefacialshrug,andconcludedwith“Iguess…theytakehim.”Theshrugfunctionedtodemonstrate
thatthespeakerwasnotcertainwhetherherconclusionwasactuallywhathadhappened,butitwasgoodenough.
Thinkingfacesandfacialshrugsfunctionascollateralcommunication(ratherthanpersonalreactions)because
theycommentonwhatthespeakerissayingratherthanprovidingtopicalcontent.
Clicktoviewlarger
Figure9 .ThespeakerwascompletingherdescriptionofascenefromShrek2when,insteadofcontinuing
onfromframe1,shemadeafacialshrug(inframe2).Thenshesaid“Iguess”(inframe3)andwenton.
Thethreeframeshotscovera1.3-secondperiod.(Thefaceoftheaddressee,whowassittingacrossatable
fromthespeaker,isinsetattheupperright.)
FacialBackchannels
Addresseesaswellasspeakersmakefacialgesturesthatfunctionascollateralcommunication.Themajorityof
addressees’facialactionsinChovil’s(1989,1991/1992)datawerebackchannelresponses.Chovilfoundthat
facialbackchannelstypicallyconsistedofbrowraises,mouthcornersturneddown,orlipspressedtogether.Recall
thatChovildidnotstudysmiles,butBrunner’s(1979)intensiveanalysisofbackchannelsrevealedthat
[Addressees’]smilebeginningsshowedastrongtendencytooccuratthesamekindsoflocationas“back
channel”responses(suchas“yeah,”“uh-huh,”andheadnods).Thisfindingindicatesthatthesmilecan
functionasatypeofbackchannel.Itisarguedthatsmiles,likeotherformsofbackchannel,make
communicationmoreefficientbyprovidingthespeakerwithfeedbackonanumberoflevels
Page 15 of 22
PRINTED FROM OXFORD HANDBOOKS ONLINE (www.oxfordhandbooks.com). (c) Oxford University Press, 2014. All Rights
Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a title in Oxford
Handbooks Online for personal use (for details see Privacy Policy).
Subscriber: Oxford University Press - Master Gratis Access; date: 05 August 2014
Hand and Facial Gestures in Conversational Interaction
simultaneously.(p.728)
Wewouldaddthataddressees’facialgesturesareefficientforanotherreasonaswell:theyproviderapid
feedbacktothespeakerwithoutinterruptingortakingupaturn.
FacialGesturesandWords
Asmanyoftheseexamplesshow,conversationalfacialgesturesaremeaningfullyrelatedtothewordstheyoccur
with,buttheyarenotalwaysredundantwiththem.Instead,facescancontributeuniqueinformationtothe
integratedmessage.Withinoneofhermajorgroups,inwhichthespeaker’sfaceconveyedinformationaboutthe
topic,Chovil(1989,1991/1992;seealsoBavelas&Chovil,1997)analyzedhowoftenthefaceprovided
nonredundantinformation(i.e.,notatallintheaccompanyingwords)versushowmanywereatleastsomewhat
redundantwithwords.Asurprising40percentofthisgroupwerecompletelynonredundant;thatis,theyconveyed
informationthatcomplementedbutwasnotconveyedintheaccompanyingwords.
ConversationalFactorsthatAffectFacialGestures
Asshownintheprecedingparagraphs,theinterlocutors’facesarequiteactiveduringface-to-facedialogue,
conveyingbothtopicalcontentandcollateralcommunication.Inviewofthisubiquity,thelackofexperimental
researchonfacialgesturesindialoguecanhardlybeoveremphasized(seereviewinBavelas&Chovil,2006).To
ourknowledge,thepublishedreportsofdialogicdataconsistofthethreesystematicanalysescitedearlier
(Brunner,1979;Chovil,1989,1992/1992;Ekman,1979)andtwoexperiments(Chovil,1989,1991;Bavelasetal.
2014).
EffectsofVisibilityonAddressees’MotorMimicry
ThefirstcontrolledexperimentonfacialgesturesindialoguewasChovil’s(1989,1991)studyofaddressees’motor
mimicry.Asdescribedearlier,motormimicryoccurswhensomeone’sreactionisnotappropriatetohisorherown
situation;instead,itisappropriatetothesituationoftheotherperson.Bavelas,Black,Lemery,andMullett(1986)
hadshownthatfacialmotormimicrydependedoneyecontactwiththeinjuredperson,buttheirexperimental
settingwasnotadialogue.Chovilcreatedfourexperimentalconditions:theaddresseewashearingaclose-call
storyeitherinaface-to-facedialogue,inadialogueonthetelephone,inadialoguethroughapartition,orwhen
alone(listeningtoanansweringmachine).Thespeakerstoldtheirownstoriesinthefirstthreeconditions;thestory
ontheansweringmachinewaschosenfromearlierdataasaparticularlyalarmingone.Therateofmimicrywas
significantlyhigherintheface-to-facedialoguethanintheotherthreeconditions,whereanaddresseewouldnot
seeit.Theratewasalsosignificantlylowerwhentheaddresseewasalone,comparedtothethreedialogue
conditions.
Altogether,theseriesofexperimentsonmotormimicryincludingBavelasetal.(1986),Chovil(1989,1991),
Bavelas,Black,Chovil,Lemery,&Mullett(1988),andBavelas,Coates,andJohnson(2000)ledtotheconclusion
thatmotormimicryisnotareflexiveoremotionalreaction;itfunctionscommunicativelyindialogue,specifically,as
adisplayofunderstanding.(SeeBavelas,2007,foracriticalreviewofsubsequentinterpretationsofthiswork.)
EffectsofVisibilityandDialogueonSpeakers’FacialPortrayals
Toourknowledge,Bavelasetal.(2014)isthefirstformalexperimentonspeakers’facialgestures,specifically,
speakers’facialportrayalsofcharactersinthemovie,Shrek2(asillustratedthroughoutthischapter).Thethree
experimentalconditionswereface-to-facedialogue,telephonedialogue,andmonologueintoamicrophone,
makingitpossibletoassesstheeffectsofbothvisibilityanddialogueseparately.Visibilityincreasedtherateof
facialportrayals(i.e.,itwassignificantlyhigherintheface-to-faceconditionthaninthetelephoneandtape
recorderconditionscombined).Dialoguealsoincreasedtherateoffacialportrayals(i.e.,itwashigherinthe
combinedface-to-faceandtelephonedialoguesthaninmonologue).NotethatChovil’s(1989,1991)experimenton
addressees’motormimicryalsoshowedbothvisibilityanddialogueeffects.
SummaryofFacialGestures
Page 16 of 22
PRINTED FROM OXFORD HANDBOOKS ONLINE (www.oxfordhandbooks.com). (c) Oxford University Press, 2014. All Rights
Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a title in Oxford
Handbooks Online for personal use (for details see Privacy Policy).
Subscriber: Oxford University Press - Master Gratis Access; date: 05 August 2014
Hand and Facial Gestures in Conversational Interaction
Incontrasttohandgestures,experimentalresearchonfacialgesturesisrare,probablyoverwhelmedbyresearch
onemotionalexpressions.Themajordescriptivestudieswerepublisheddecadesago,but,asshowninthis
chapter,theyfitnewdatajustaswell.Giventhatfacesindialoguearelively,eloquent,efficient,infinitelyvaried,
andubiquitous,itiswellpasttimetostartinvestigatingtheminmoredetail.
Conclusion
Face-to-facedialogueisthebasicformoflanguageuse(e.g.,Clark,1996;Goodwin,1981;Levinson,1983).The
processesbywhichinterlocutorsindialoguecoordinatetheiractionstocreatemeaninghavebeguntointerest
scholarsfrommanydisciplines,includinglinguistics,anthropology,andpsychology.Researchonhowinterlocutors
usevisibleresources,suchashandandfacialgestures,isintrinsictounderstandinglanguageuseinface-to-face
dialogue.Regardlessofthediscipline,investigationsoflanguageuseindialoguemustmeet,atminimum,two
methodologicalimperatives.First,theinteractionsthemselvesmustmaintaintheessentialfeaturesofdialogue,
includinginterlocutors’abilitytobespontaneousandtoextemporize.Thestudiesreviewedheredemonstratethat
thisimperativecaneasilybemetinanexperimentalsettingwhennecessarycontrolisachievedthroughcareful
researchdesignratherthanthroughreductiontoaquasi-dialogueor,asweputitelsewhere,committing
“dialogicide”(Gerwing&Bavelas,2013).
Thesecondmethodologicalimperativeisthattheoriginaldatamustnotonlyberecordedfull-faceonvideo,butthe
analysismustalsobeconducteddirectlyfromvideo.Handandfacialgesturescoordinatewithandcomplement
speech,andtheyoftenprovideessentialdetailsthatarenotinthewords.Digitizedvideoandsuitablesoftware
(e.g.,ELAN,http://tla.mpi.nl/tools/tla-tools/elan/;Wittenburgetal.,2006)permitthecloseobservationofdialogueas
alivelysynchronyofwords,gestures,andfaces.Withframe-by-frame,repeatedviewing,analystscanobserve
interlocutors’behaviorscarefullyandsystematically.Rigorousproceduresforanalysisanddetailedoperational
definitions,plusregulartestsforinteranalystreliabilityareessentialforensuringinteranalystagreement.
Theoverviewpresentedinthischapterleadstoseveralbroadconclusions:handgesturesarenotsimply
redundantmovementsnorsignsofdysfluency.Theycontributeawidevarietyofinformationrangingfromthe
concreteandphysicaltothemetaphoricaltometa-communicativecollateralcommunication.Theevidence
suggeststhatthisfieldhasmaturedfromadebateoverwhethergesturesarecommunicativeintosolidresearchon
howtheycommunicate.Norarefacialgesturesalimited,staticsetofemotionalconfigurations.Theyarea
constantlychangingaccompanimenttowords,addingnewandnuanceddetails,againrangingfromtheconcrete
totheabstracttocollateralcommunication.
Anabundantandincreasingnumberofquantitativeexperimentsonhandgesturesareexploringsubtle
relationshipsandunexpectedtopics.Inshort,thisfieldishealthyandgrowingrapidly.However,thestudyoffacial
gesturesneedstoemergefromthedominanceofemotiontheoryinordertoproduceabodyofresearchofitsown.
Researchonthecontributionsthatinterlocutorsmakewiththeirfacescannotadvanceuntilface-to-facedialogues
areroutinelyrecordedandmicroanalyzed.Foryoungresearchers,thisfieldiswideopenforsystematic
descriptionandespeciallyforexperimentalresearch.Everyareatouchedoninthischaptercouldexpandmany
timesover.Recentresearchonhandgestureprovidesmodelsandmethodsthatmightapplytostudiesoffacial
gestureaswell.
Onereservationisthatexperimentalworkhastoooftentreatedgestures,faces,andwordsasseparable
phenomena.Inface-to-facedialogue,these(andother)communicativeresourcesworktogether,anditisessential
toexplorethedetailsoftheirdivisionoflabor.Thesemanticfeaturesapproach(e.g.,Beattie&Shovelton,1999;
Gerwing&Allison,2009)isonepossiblemethodforquantifyingthecontributionsofeachmodalitybyprovidinga
systematicapproachforexploringhowvariationsinsocialcontext(e.g.,mutualvisibility,commonground)might
influencethedistributionofinformationamonghandgestures,facialgestures,andwords.Another,mostly
unexploredareaishowallofthesecometogethertopresentaunitarymultimodalconfiguration,suchastheone
emphasizedinFigure1.Moreover,languageuseinface-to-facedialogueismorethanwords,handgestures,and
facialgestures.Prosodyandgazesuggestthemselvesasnewareasforresearch—providedthatthesestudies
alwaystakeplacewithintrueface-to-facedialogues.Scrapsofscriptedtalkandrecordingsofgazeatinanimate
targetsfittheoutmodedprincipleofreductionismthatcontinuestothreatenthestudyofconversationsinface-tofacedialogue.
Page 17 of 22
PRINTED FROM OXFORD HANDBOOKS ONLINE (www.oxfordhandbooks.com). (c) Oxford University Press, 2014. All Rights
Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a title in Oxford
Handbooks Online for personal use (for details see Privacy Policy).
Subscriber: Oxford University Press - Master Gratis Access; date: 05 August 2014
Hand and Facial Gestures in Conversational Interaction
Otherthanafewexemplarsreviewedhere(Bavelasetal.,2011;Clark&Krych,2004;Holler&Wilkin,2011;
Furuyama,2000),researchershavetendedtoignorethesequentialrelationshipsbetweeninterlocutors’behaviors
astheirdialogueunfolds.Thishesitationhasheldthefieldbackfromunderstandinghowvisiblebehaviorsare
sequentiallyintegratedintothedialogueandhowtheyinfluencesubsequentutterances.Onewell-established
modelforstudyingtheserelationshipswithinanexperimentalsettingisareferentialcommunicationtask(e.g.,
Holler&Wilkin,2011),inwhichpairsofparticipantsmustcompleteaseriesofstepstogetherbutonepersonhas
allofthenecessaryinformationandtheotherpersondoesnot.Althoughusedextensivelytostudythesequential
processesofverbalcommunication,thismethodisunderusedforexploringhowinterlocutorsusetheirhandand
facialgesturessequentially.
Onatheoreticallevel,ourresearchgroupispursuingatheoryofdemonstrations(Bavelasetal.,2008,2014),
whichincludenotonlyhandandfacialgesturesbutalsodirectquotationsandmetaphors.Weproposethat
demonstrationsrequireanaddresseeandarethereforeuniquelylinkedtointeractingindialogueratherthan
monologue.Underpinningthistheoryistheproposalthat,ratherthandividingtheelementsofdialoguealong
physicallinessuchas“verbal”and“nonverbal,”itmightbemoreusefultomakefunctionaldistinctions,suchas
betweendescriptionanddemonstration.Exactlyhowdemonstrationsworkalongsidedescriptivespeechis
anotherwide-openquestion.
Throughoutthischapter,wehaveemphasizedexperimentalresearch,notbecauseitissuperiortoothermethods,
butbecauseitisausefulandinformativemethodthatismostoftenusedinlanguageandsocialpsychology.It
wouldberemissnottopointoutthatmostoriginalinsightscomefromeverydayobservationsandqualitative
analyses(e.g.,Bavelas,1987).Theseprovideaspringboardforresearchquestionsandhypotheses,whichleadto
designingexperimentsthatmightanswerortestthem.Thereisaworldofunexploreddialogicphenomenaout
there,onethatisnotfoundinlibrariesorstatistics.
References
Alibali,M.W.,Heath,D.C.,&Myers,H.J.(2001).Effectsofvisibilitybetweenspeakerandlistenerongesture
production:Somegesturesaremeanttobeseen.JournalofMemoryandLanguage,44,169–188.
Bavelas,J.B.(1987).Permittingcreativityinscience.InD.N.Jackson&J.P.Rushton(Eds.),Scientificexcellence:
Originsandassessment(pp.307–327).BeverlyHills:Sage.
Bavelas,J.B.(2005).Thetwosolitudes:ReconcilingSocialPsychologyandLanguageandSocialInteraction.InK.
Fitch&R.Sanders(Eds.),Handbookoflanguageandsocialinteraction(pp.179–200).Mahwah,NJ:Erlbaum.
Bavelas,J.B.(2007).Face-to-facedialogueasamicro-socialcontext.Theexampleofmotormimicry.InS.
Duncan,E.Levy,&J.Cassell(Eds.),Gestureandthedynamicdimensionoflanguage(pp.127–146).Amsterdam:
JohnBenjamins.
Bavelas,J.B.,Black,A.,Chovil,N.,Lemery,C.R.,&Mullett,J.(1988).Formandfunctioninmotormimicry.
Topographicevidencethattheprimaryfunctioniscommunicative.HumanCommunicationResearch,14,275–299.
Bavelas,J.B.,Black,A.,Lemery,C.R.,&Mullett,J.(1986).“Ishowhowyoufeel.”Motormimicryasa
communicativeact.JournalofPersonalityandSocialPsychology,50,322–329.
Bavelas,J.B.,&Chovil,N.(1997).Facesindialogue.InJ.A.Russell&J.M.Fernandez-Dols(Eds.),Thepsychology
offacialexpression(pp.334–346).Cambridge,UK:CambridgeUniversityPress.
Bavelas,J.B.,&Chovil,N.(2000).Visibleactsofmeaning.Anintegratedmessagemodeloflanguageuseinfaceto-facedialogue.JournalofLanguageandSocialPsychology,19,163–194.
Bavelas,J.B.,&Chovil,N.(2006).Handgesturesandfacialdisplaysaspartoflanguageuseinface-to-face
dialogue.InV.Manusov&M.Patterson(Eds.),Handbookofnonverbalcommunication(pp.97–115).Thousand
Oaks,CA:Sage.
Bavelas,J.B.,Chovil,N.,Coates,L.,andRoe,L.(1995).Gesturesspecializedfordialogue.PersonalityandSocial
Page 18 of 22
PRINTED FROM OXFORD HANDBOOKS ONLINE (www.oxfordhandbooks.com). (c) Oxford University Press, 2014. All Rights
Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a title in Oxford
Handbooks Online for personal use (for details see Privacy Policy).
Subscriber: Oxford University Press - Master Gratis Access; date: 05 August 2014
Hand and Facial Gestures in Conversational Interaction
PsychologyBulletin,21,394–405.
Bavelas,J.B.,Chovil,N.,Lawrie,D.A.,&Wade,A.(1992).Interactivegestures.DiscourseProcesses,15,469–489.
Bavelas,J.B.,Coates,L.,&Johnson,T.(2000).Listenersasco-narrators.JournalofPersonalityandSocial
Psychology,79,941–952.
Bavelas,J.B.,Gerwing,G.,Allison,M.,&Sutton,C.(2011).Dyadicevidenceforgroundingwithabstractdeictic
gestures.InG.Stam,M.Ishino,&R.Ashley(Eds.),Integratinggestures:Theinterdisciplinarynatureofgesture
(pp.49–60).Amsterdam:Benjamins(GestureStudiesseries).
Bavelas,J.B.,Gerwing,J.,&Healing,S.(2014).Theeffectofdialogueondemonstrations:Directquotations,facial
portrayals,handgestures,andfigurativereferences.DiscourseProcesses,inpress.
Bavelas,J.B.,Gerwing,J.,Sutton,C.,&Prevost,D.(2008).Gesturingonthetelephone:Independenteffectsof
dialogueandvisibility.JournalofMemoryandLanguage,58,495–520.
Bavelas,J.B.&Healing,S.(2013).Reconcilingtheeffectsofmutualvisibilityongesturing.Gesture13(1),63–92.
Beattie,G.,&Shovelton,H.(1999).Doiconichandgesturesreallycontributeanythingtothesemanticinformation
conveyedbyspeech?Anexperimentalinvestigation.Semiotica,123,1–30.
Brunner,L.J.(1979).Smilescanbebackchannels.JournalofPersonalityandSocialPsychology,37,728–734.
Buchler,J.(Ed.).(1940).ThephilosophyofPeirce:Selectedwritings.London:Routledge&KeganPaul.
Chovil,N.(1989).Communicativefunctionsoffacialdisplaysinconversation.UnpublishedPh.D.dissertation,
UniversityofVictoria,Victoria,BC.
Chovil,N.(1991).Socialdeterminantsoffacialdisplays.JournalofNonverbalBehavior,15,141–153.
Chovil,N.(1991/1992).Discourse-orientedfacialdisplaysinconversation.ResearchonLanguageandSocial
Interaction,25,163–194.
Chovil,N.(2005).Measuringconversationalfacialdisplays.InV.Manusov(Ed.),Thesourcebookofnonverbal
measures:Goingbeyondwords.Hillsdale,NJ:LawrenceErlbaum.
Clark,H.H.(1985).Languageandlanguageusers.InG.Lindzey&E.Aronson(Eds.),Thehandbookofsocial
psychology(3rded.,vol.2,pp.179–232).NewYork:RandomHouse.
Clark,H.H.(1996).Usinglanguage.Cambridge:CambridgeUniversityPress.
Clark,H.H.,&Gerrig,R.J.(1990).Quotationsasdemonstrations.Language,66,764–805.
Clark,H.H.&Krych,M.A.(2004).Speakingwhilemonitoringaddresseesforunderstanding.JournalofMemoryand
Language,50(1),62–81.
Clark,H.H.,&Wilkes-Gibbs,D.(1986).Referringasacollaborativeprocess.Cognition,22,1–39.
Cohen,A.A.(1977).Thecommunicativefunctionsofhandillustrators.JournalofCommunication,27,54–63.
Cohen,A.A.,&Harrison,R.P.(1973).Intentionalityintheuseofhandillustratorsinface-to-facecommunication
situations.JournalofPersonalityandSocialPsychology,28(2),276–279.
Danziger,K.(1990).Constructingthesubject.Historicaloriginsofpsychologicalresearch.Cambridge(UK):
CambridgeUniversityPress.
deRuiter,J.P.,Bangerter,A.,&Dings,P.(2012).Theinterplaybetweengestureandspeechintheproductionof
referringexpressions:Investigatingthetradeoffhypothesis.TopicsinCognitiveScience4(2),232–248.
Ekman,P.(1977).Biologicalandculturalcontributionstobodyandfacialmovement.InJ.Blacking(Ed.),
AssociationofSocialAnthropologistsmonograph15,Theanthropologyofthebody.London:AcademicPress.
Page 19 of 22
PRINTED FROM OXFORD HANDBOOKS ONLINE (www.oxfordhandbooks.com). (c) Oxford University Press, 2014. All Rights
Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a title in Oxford
Handbooks Online for personal use (for details see Privacy Policy).
Subscriber: Oxford University Press - Master Gratis Access; date: 05 August 2014
Hand and Facial Gestures in Conversational Interaction
Ekman,P.(1979).Aboutbrows:Emotionalandconversationalsignals.InM.vonCranach,K.Foppa,W.Lepenies,&
D.Ploog(Eds.),Humanethology(pp.169–249).Cambridge,UK:CambridgeUniversityPress.
Ekman,P.(1997).Shouldwecallitexpressionorcommunication?EuropeanJournalofSocialSciences,10,333–
359.
Ekman,P.,&Friesen,W.V.(1978).TheFacialActionCodingSystem.PaloAlto,CA:ConsultingPsychologists
Press.
Ekman,P.E.(1985).Tellinglies.NewYork:BerkleyBooks.
Emmorey,K.,&Casey,S.(2001).Gesture,thoughtandspatiallanguage?Gesture,1,35–50.
Fowler,C.A.(1988).Differentialshorteningofrepeatedcontentwordsproducedinvariouscommunicative
contexts.LanguageandSpeech,31(4),307–319.
Fridlund,A.J.,Ekman,P.,&Oster,H.(1987).Facialexpressionsofemotion:Reviewofliterature,1970–1983.InA.
W.Siegman&S.Feldstein(Eds.),Nonverbalbehaviorandcommunication(2nded.,pp.143–224).Hillsdale,NJ:
Erlbaum.
Furuyama,N.(2000).Gesturalinteractionbetweentheinstructorandthelearnerinorigamiinstruction.InD.McNeill
(Ed.),Languageandgesture(pp.99–117).Cambridge:CambridgeUniversityPress.
Gerwing,J.,&Allison,M.(2009).Therelationshipbetweenverbalandgesturalcontributionsinconversation:A
comparisonofthreemethods.Gesture,9(3),313–336.
Gerwing,J.,&Allison,M.(2011).Theflexiblesemanticintegrationofgesturesandwords.Comparingface-to-face
andtelephonedialogues.Gesture,11(3),308–329.
Gerwing,J.,&Bavelas,J.B.(2004).Linguisticinfluencesongesture’sform.Gesture,4(2),157–195.
Gerwing,J.,&Bavelas,J.(2013).Thesocialinteractivenatureofgestures:Theory,assumptions,methods,and
findings.InC.Müller,A.Cienki,E.Fricke,S.H.Ladewig,D.McNeill,&S.Tessendorf(Eds.),Body-languagecommunication,Volume1,Contemporaryapproaches(Ch.51,pp.821-836).Berlin:MoutonDeGruyter.
Goodwin,C.(1981).Conversationalorganization:Interactionbetweenspeakersandhearers.NewYork:
AcademicPress.
Gullberg,M.(2010).MethodologicalreflectionsongestureanalysisinSLAandbilingualismresearch.Second
LanguageResearch,26(1),75-102.
Haviland,S.E.,&Clark,H.H.(1974).What’snew?Acquiringnewinformationasaprocessincomprehension.
JournalofVerbalLearningandVerbalBehavior,13,512–521.
Healing,S.,&Gerwing,J.(2012July).Spatialcohesivenessingesturesequences.Effectsofmutualvisibility.
PaperpresentedatthemeetingoftheInternationalSocietyforGestureStudies,Lund(Sweden).
Holler,J.,&Stevens,R.(2007).Theeffectofcommongroundonhowspeakersusegestureandspeechto
representsizeinformation.JournalofLanguageandSocialPsychology,26(1),4–27.
Holler,J.,Tutton,M.,&Wilkin,K.(2011).Co-speechgesturesintheprocessofmeaningcoordination.In
Proceedingsofthe2ndGESPIN—GestureinSpeechandInteractionConference,Bielefeld,5–7,September2011.
Holler,J.,&Wilkin,K.(2011).Co-speechgesturemimicryintheprocessofcollaborativereferringduringface-tofacedialogue.JournalofNonverbalBehavior,35,133–153.
Kendon,A.(1980).Gesticulationandspeech:Twoaspectsoftheprocessofutterance.InM.R.Key(Ed.),The
relationshipofverbalandnonverbalcommunication(pp.207–227).TheHague:Mouton.
Kendon,A.(2004).Gesture.Visibleactionasutterance.Cambridge,UK:CambridgeUniversityPress.
Page 20 of 22
PRINTED FROM OXFORD HANDBOOKS ONLINE (www.oxfordhandbooks.com). (c) Oxford University Press, 2014. All Rights
Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a title in Oxford
Handbooks Online for personal use (for details see Privacy Policy).
Subscriber: Oxford University Press - Master Gratis Access; date: 05 August 2014
Hand and Facial Gestures in Conversational Interaction
Krauss,R.M.,Dushay,R.A.,Chen,Y.,&Rauscher,F.(1995).Thecommunicativevalueofconversationalhand
gestures.JournalofExperimentalSocialPsychology,31(6),533–552.
Kraut,R.E.,&Johnston,R.E.(1979).Socialandemotionalmessagesofsmiling:Anethologicalapproach.Journal
ofPersonalityandSocialPsychology,37,1539–1553.
Kuhlen,A.K.,&Brennan,S.E.(2012).Languageindialogue:Whenconfederatesmightbehazardoustoyourdata.
AcceptedforpublicationatPsychonomicBulletin&Review.doi:10.3758/s13423-012-0341-8.Thefinal
publicationisavailableatwww.springerlink.com.
Levinson,S.C.(1983).Pragmatics.Cambridge:CambridgeUniversityPress.
Luria,A.R.(2004).Reductionisminpsychology.InR.L.Gregory(Ed.),TheOxfordcompaniontothemind(2nd
ed.,pp.675–676).Oxford,England:OxfordUniversityPress.
McNeill,D.(1985).Soyouthinkgesturesarenonverbal?PsychologicalReview,92,350–371.
Mol,L.,Krahmer,E.,Maes,A.,&Swerts,M.(2009a).Communicativegesturesandmemoryload.InProceedingsof
the31stAnnualConferenceoftheCognitiveScienceSociety,Amsterdam,Netherlands,1569–1574.
Mol,L.,Krahmer,E.,Maes,A.,&Swerts,M.(2009b).Thecommunicativeimportofgestures.Evidencefroma
comparativeanalysisofhuman-humanandhuman-machineinteractions.Gesture,9(1),97–126.
Özyürek,A.(2000).Theinfluenceofaddresseelocationonspatiallanguageandrepresentationalgesturesof
direction.InD.McNeill(Ed.),Languageandgesture(pp.64–83).Cambridge,UK:CambridgeUniversityPress.
Özyürek,A.(2002).Dospeakersdesigntheirco-speechgesturesfortheiraddressees?Theeffectsofaddressee
locationonrepresentationalgestures.JournalofMemoryandLanguage,46(4),688–704.
Parrill,F.(2008).Subjectsinthehandsofthespeakers:Anexperimentalstudyofsyntacticsubjectandspeechgestureintegration.CognitiveLinguistics,19,283–299.
Pine,K.J.,Burney,D.J.,&Fletcher,B.(2010).Thesemanticspecificityhypothesis:Whengesturesdonotdepend
uponthepresenceofalistener.JournalofNonverbalBehavior,34,169–178.
Reber,A.S.,Allen,R.,&Reber,E.S.(2009).Penguindictionaryofpsychology(4thed.).London,UK:Penguin.
Rimé,B.(1982).Theeliminationofvisiblebehaviorfromsocialinteractions:Effectsonverbal,nonverbaland
interpersonalvariables.EuropeanJournalofSocialPsychology,12,113–129.
Rowbotham,S.,Holler,J.,Lloyd,D.,&Wearden,A.(2011).Howdowecommunicateaboutpain?Asystematic
analysisofthesemanticcontributionofco-speechgesturesinpain-focusedconversations.JournalofNonverbal
Behavior,36(1),1–21.
Schober,M.F.,&Clark,H.H.(1989).Understandingbyaddresseesandoverhearers.CognitivePsychology,21,
211–232.
Sears,R.R.(1951).Atheoreticalframeworkforpersonalityandsocialpsychology.AmericanPsychologist,6,476–
483.
Thibaut,J.W.,&Kelley,H.H.(1959).Thesocialpsychologyofgroups.NewYork:Wiley.
Wittenburg,P.,Brugman,H.,Russel,A.,Klassman,A.,&Sloetjes,H.(2006).ELAN:Aprofessionalframeworkfor
multimodalityresearch.In:ProceedingsofLREC2006,FifthInternationalconferenceonlanguageresourcesand
evaluation,Genoa.
Notes:
(1)KrautandJohnstonusedthetermfacialdisplay,borrowedfromthestudyofanimalbehavior,wheretheovert
Page 21 of 22
PRINTED FROM OXFORD HANDBOOKS ONLINE (www.oxfordhandbooks.com). (c) Oxford University Press, 2014. All Rights
Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a title in Oxford
Handbooks Online for personal use (for details see Privacy Policy).
Subscriber: Oxford University Press - Master Gratis Access; date: 05 August 2014
Hand and Facial Gestures in Conversational Interaction
behavioractsasasocialdisplaytoothers.Chovilalsoadoptedthisterminherextensivework.Weproposea
changetofacialgesture,whichemphasizesthemanysimilaritiestohandgestures.
JanetBavelas
JanetBavelas,DepartmentofPsychology,UniversityofVictoria
JenniferGerwing
JenniferGerwing,DepartmentofPsychology,UniversityofVictoria
SaraHealing
SaraHealing,DepartmentofPsychology,UniversityofVictoria
Page 22 of 22
PRINTED FROM OXFORD HANDBOOKS ONLINE (www.oxfordhandbooks.com). (c) Oxford University Press, 2014. All Rights
Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a title in Oxford
Handbooks Online for personal use (for details see Privacy Policy).
Subscriber: Oxford University Press - Master Gratis Access; date: 05 August 2014