Examining the Kinetic Changes in Pupillary Response and Listening Effort Senior Thesis Presented to The Faculty of the School of Arts and Sciences Brandeis University Undergraduate Program in Neuroscience Dr. Arthur Wingfield, Advisor By Austin Luor May, 2017 Copyright by Austin Luor 1 Table of Contents Abstract Chapter 1: Examining the Kinetic Changes in Pupillary Response and Listening Effort A. Introduction 1. Reflex pupillary response to light 2. Pupillary response to cognitive effort 3. Research motivation 4. Summary and hypotheses B. Methods C. Results D. Discussion Chapter 2: Effects of Healthy aging on pupillary kinetics in response to auditory stimuli that vary in complexity: A pilot study A. Introduction 1. Effects of aging on pupillary light reflex 2. Effects of aging on pupillometry in response to cognitive efforts 3. Research motivation 4. Hypotheses B. Preliminary Results and Discussion C. Conclusions D. Acknowledgments E. Appendices F. References 2 ABSTRACT Pupil dilation as a measure of cognitive effort in listening tasks has been well documented (Kahneman & Beatty 1966; Kuchinsky et al. 2013; Piquado et al. 2010; Wingfield et al. 2015; Winn et al. 2016). However, a direct comparison of the pupillary dynamics while attending to different types of auditory tasks has not been investigated; nor has the nature of pupillary response elicited by processing simple auditory stimuli (tones) versus more complex speech stimuli (words, sentences) been explored. In this thesis, I examine several parameters of changes in young adults’ pupil size, including peak amplitude and rate of increase to peak, elicited while making decisions about auditory stimuli that varied in acoustic or linguistic complexity. Results showed that peak pupil amplitude was the greatest in a lexical decision task, smaller for a sentence decision task, and the smallest in a tone decision task. The sentence decision task resulted in a longer duration to peak compared to tone decision and lexical decision tasks. These results suggest that differences in listening/decision tasks influence the dynamics of the pupillary response, which may suggest differences in processing effort. A pilot study that examines older adults’ pupil size while attending to the same auditory decision tasks is included in this thesis as an indication of future direction this work will take. 3 Chapter 1 Examining the Kinetic Changes in Pupillary Response and Listening Effort INTRODUCTION The Pupillary response (changes to the size of the pupils of the eyes) is well known for its response to ambient light. Known as the pupillary light reflex, pupils constrict in response to an increase in ambient light and dilate when there is a decrease in light. Such activity reflects the involuntary physiological function that regulates the amount of light that reaches the retina. In addition to its primary physiological function, researchers have also found changes to the pupil diameter offers a unique window on perceptual and cognitive effort expended during mental tasks. Studies has shown that the pupil also dilates in response to arousal and affect (Kahneman, 1973; Naber et al. 2013). Important to this thesis, studies have also shown that pupillary responses can be used to index the extent of central nervous system processing allocated to mental tasks (Beatty, 1982). Such task-evoked pupil dilation is related to the activation of the Locus coeruleus-norepinephrine system and has been systematically observed in response to cognitive tasks that measure working memory load (Kahneman & Beatty, 1966), and attention allocation (Verney et al., 2004). Pupillometry has also been utilized to measure cognitive effort in response to auditory tasks. Most notably, this method has been used to index effort in participants’ responses to auditorily presented stimuli such as digits, words, sentences (e.g., Piquado et al., 2010; Zekveld et al., 2011; Kuchinsky et al., 2013). The use of pupillometry has became popular because the 4 degree of pupil dilation appears to correspond to the tasks that thought to require greater cognitive processing, (e.g recalling longer word lists > recalling shorter word lists, comprehending syntactically more complex sentences > comprehending syntactically less complex sentences) and it is a measure that participants cannot voluntarily control. Jackson Beatty (1982) theorized that the task-evoked pupillary response provides a reliable and sensitive indication of within-task, across-task, and across population variations in mental processing load. Despite the wide use of pupillometry in the field of cognitive psychology at hearing science, no studies to our knowledge have specifically compared the pupil response to one type of auditory stimulus versus different types of stimuli. The purpose of this current study is to close this gap in knowledge by examining pupillometry in the context of several different tasks and stimuli that people have used across many studies. This thesis investigates the use of pupillometry as a measure of cognitive effort. This thesis will focus on young adults and examine their pupillary measurement parameters elicited by auditory stimuli that vary in presumed complexity. My overall goal of this study, however, is to understand the extent to which pupillometric methods can be used as a psychophysiological measure of the effects of aging. The successful outcome of this study may serve as a baseline for future work using pupillometry as a measure of listening effort in the study of cognitive aging. As an indication of the direction for future work, I include a pilot study in the appendix that compares the dynamics of pupillary response of young and older adults in response to various cognitive tasks. 5 Reflexive Pupillary Response to Light General The pupillary response is characterized by the changes to the size of the pupils of eyes via the optic and oculomotor cranial nerves. Known as the pupillary light reflex, pupils constrict to an increase in ambient light and dilate when there is a decrease in light to regulate the amount of incoming light that lands on the retina. The pupillary light reflex is under direct control of the autonomic nervous system and is a reflection of the balance between the sympathetic nervous system (SNS) and the parasympathetic nervous system (PNS) (Wang et al., 2016). When presented with ambient light, the pupillary light reflex pathway involves a systematic 1) fast constriction phase, followed by 2) a fast redilation phase, and then 3) a slow redilation phase where the pupil recovers to its original size. A maximum constriction peak separates the initial fast constriction phase and the fast redilation phase. A maximum dilation peak is reached after the slow redilation stage as the pupil recovers to its original size. According to Bitsios (1996) and Steinhauer (2004), the resting pupil diameter reflects the balance between the opposing parasympathetic constrictor and the sympathetic dilator influences. The PNS plays a primary role during the pupil constriction phase, while the SNS contribution is negligible. On the contrary, both the PNS and the SNS innervate the pupil in the beginning of the redilation phase, and the PNS is negligible during the dilation phase (Loewenfeld and Lowenstein 1999). Thus, in theory, a quantification of the pupillary constriction in the pupil light reflex pathway is an indication of PNS activity uncontaminated by SNS activity (Wang et al., 2016). In general, reduced PNS activity is characterized by longer constriction latency, slower maximum constriction velocity, and smaller constriction amplitude of the pupil light reflex (Loewenfeld & Lowenstein 1999) while reduced SNS activity is 6 characterized by reduced pupil diameters, reduced darkness reflex amplitude and dilation velocities, and prolonged recovery times of PLR (Smith et al, 1992). Pupillometry in cognitive effort In addition to the pupillary light reflex, changes in the pupil size occur due to perceptual and cognitive effort have also been observed. It has been long known in the neurophysiological literature that pupil dilates during cognitive activity-- i.e., the greater the effort expended, the larger the pupil size (Beatty 1982, Hess & Polt 1964, Kahneman & Beatty 1966). Past studies have shown that pupil diameter is a sensitive, indirect measure of task-evoked cognitive processing load in various cognitive tasks. Among the earliest tasks that used pupillary dilations as an index of cognitive load was mental arithmetic, specifically, mental multiplication (Hess & Polt 1964). Pupil dilations were not only observed during mental multiplication, but also observed in tasks associated with working memory. Kahneman demonstrated that pupil size increased incrementally as items of a to-be-recalled list of digits were heard and stored in memory (Kahneman & Beatty 1966). Kahneman suggested that the size of pupillary dilations correlated with the current load on working memory in simple tasks requiring short term retention of a sequence of digits. This was the first study that demonstrated the interplay between increasing complexity of a memory task and greater elicited pupil dilation as an index of cognitive effort. Consequently, researchers applied this measurable change in the pupil of the eye in response to increasing mental activity to assay the cognitive load including: spatial processing (Just & Carpenter, 1993), speech perception (Just & Carpenter & Miyake, 2003), speech 7 production (Hoeks 1995), and spoken language comprehension tasks (e.g., Kuchinsky et al., 2013; Piquado et al., 2010; Zekveld et al., 2011). Kahneman and Beatty (1966) first compared pitch discrimination with signal detection using pupillometry. They indicated that pitch discrimination involved more cognitive processing load than pitch detection alone, as the discrimination task revealed systematically larger pupil dilation than those evoked by Beatty & Lucero-Wagoner’s (2000) pitch detection task. Therefore, the next step was to investigate the pupillary changes in response to the manipulation of stimulus difficulty such as the variation in different signal to noise ratios (Zekveld et al. 2010, 2011), sentence processing (Kramer et al. 2007), and syntactic complexity (Piquado et. al. 2010). Piquado and colleagues investigated cognitive effort by comparing auditory sentence processing in young and older adults and used pupillometry to assess the effects of syntactic complexity and sentence length on processing load during listening. Additionally, Kramer et al. (1997) used sentences stimuli whereas Zekveld et al. (2010, 2011) utilized the task-evoked pupillary responses in order to investigate processing challenge of speech stimuli by varying signal to noise ratio. The aim of all these studies was to provide insight into the relation between processing demands imposed by the stimuli, and its consequent task-evoked pupillary response. Supporting the assumption the task evoked pupillary response is indicative of the cognitive effort, a larger pupillary response was observed consistently in difficult conditions than in easier conditions implying the necessity of requiring more cognitive resources. These lines of evidence have supported the fact that the task-evoked pupil response is related to the amount of cognitive resources used in an effortful sentence processing task with pupil dilation serving as a window on brain activity and function. Therefore, pupillometry serves as an important and sensitive tool 8 that has been able to indirectly assay the degree of cognitive effort in response to varying complexity and types of stimuli. Research Motivation There has been numerous studies using pupillometry to compare the relative effort required by older adults versus younger adults to perform a task such as comprehending a sentence heard in noise. A major problem is that the average pupil size of older adults is smaller than younger adults as is the size of excursion from maximum to minimum pupil size when tested using light reflex. To overcome this problem, various method has been employed such as presenting a pupil size as ratio to the individual’s maximum to light and dark field. In spite of this, there has been little attention paid to potential differences across age groups such as potential changes in velocity of eye movement. The purpose of my thesis is to close this gap in knowledge and serve as a baseline work for future pupillometry study that investigates cognitive aging where senile miosis represents a challenge in making direct comparison of processing effort in younger versus older adults. Experiment 1 explores the changes in pupil size and slope to peak for younger adults with the goal of establishing the baseline for analogous study for older adults. In this regard, I show the pilot data in chapter 2 to indicate the proposed future direction of this work. As indicated in this literature review, the task-evoked pupillary response has been used as a measure of cognitive effort for over 50 years and more recently, as a way of comparing the relative effort required by young versus older adults to accomplish the same cognitive task. My overall research motivation arises from the notion that studies which investigated task-evoked pupillary responses have not yet explored the differences in pupillary parameters and kinetics 9 among younger adults in response to a simple or a more complex listening tasks. The general hypothesis is that differences in pupillary kinetics will be elicited in younger adults making decisions in response to different levels of complexity of listening tasks. In the current study, I examine the pupillary kinetics of younger adults across three different types of auditory tasks. In this experiment, I will measure the pupillary response to three different stimulus related, two-alternative choice categorization tasks: (1) categorization of presented tone as a piano tone or not a piano tone (i.e., a pure tone), (2) word-nonword lexical decision, and (3) categorization of sentences as expected or unexpected in meaning. It has been found previously that pupil responds to a range of relatively basic auditory tasks. Kramer (2013) suggested that identifying meaningful words elicited a greater pupil dilation than elicited by listening to background noise. The present aim was to provide insight into the relation between task demand, the linguistic complexity of the stimuli (listening to sound versus listening to a meaningful word in background noise), and pupil response. Consistent with Kramer’s (2013) finding, I hypothesize that peak pupil dilation will be greater for the lexical decision task than the tone identification task due to the fact that words entail more meaning than tones. On the basis of previous literature (Kahneman & Beatty 1966, Kramer et al. 2016, Winn et al. 2016), peak pupil dilation and mean slope to peak are influenced by the types of cognitive task and the levels of task difficulty. Kahneman and Beatty (1966) observed that when participants had to recall the greatest number of digits (7 digits) in a task, a greater peak pupil dilation and a steeper slope to peak (defined as the rate of increase to peak in this study) were elicited when participants have to recall the greatest number of digits in a string. This suggested that pupillary kinetics are affected by the task difficulty and, more importantly, the number of 10 items participants have to process. Therefore, with the increasing number of words to process, I hypothesize that greater peak pupil dilation will be elicited by processing sentence stimuli than the word and tone stimuli. Consistent with Kahneman and Beatty’s observations, I also expect that the rate of increase to peak will be the fastest elicited by processing sentence stimuli compared to the word and tone stimuli due to processing greater number of words. Finally, I hypothesize that the difference in pupillary parameters will be observed in response to making judgements on meaningful versus meaningless stimuli. I expect that peak pupil dilation will be altered in response to stimuli type, specifically, by the linguistic characteristics of stimulus (whether the stimuli are meaningful or not). Past studies have investigated the pupillary activity in response to high frequency or low frequency words. Papesh and Goldinger (2012) found that hearing low frequency words evoked a greater pupil dilation than high frequency words, suggesting that low frequency words demanded greater cognitive resources to process. Therefore, I expect that meaningless stimuli (nonword and unexpected sentences) will elicit greater pupil dilation due to the low chances that such stimuli would be encountered in conversations. The peak pupil dilation will be greater for nonwords stimuli than real word stimuli and greater for unexpected sentence stimuli than expected sentence stimuli. I predict that the pupillary response to pure tones and piano tones would not differ significantly in peak pupil dilation because there is no reason to believe that pure tones and piano tones, matched in acoustic frequency, would differ in the amount of cognitive resources required to process them. 11 METHODS Participants Participants were 24 young adults (4 men, 16 women) ranging in age from 18 to 31 years (M = 22.6, SD = 3.9). The young adults were Brandeis University undergraduate and graduate students. Pure tones screening and speech reception threshold (SRT) testing was conducted using a Grason-Stadler GSI61 clinical two-channel audiometer in a sound-attenuated testing room before the start of the experiment to confirm that the participants’ hearing acuity was within age-normal limits in the frequency range considered based on SRTs and PTA across 500, 1000, 2000, and 4000 Hz. The young adults’ verbal abilities were assessed by the Shipley Vocabulary test (M younger = 13.6, SD = 1.4). All participants were native speakers of American English and provided written informed consent before participating in this study. Materials Two-alternative Forced Choice Categorization Tasks Tone identification task In this task, participant will be asked to determine whether a tone is a pure tone or a piano note. Twelve piano tones (Middle C - B; 261.6 Hz to 493.9 Hz) and twelve pure tones (matched in acoustic frequencies with piano tones) were selected. Piano tones were downloaded from the University of Iowa Electronic Music Studios’ Musical Instrument Samples Database (http://theremin.music.uiowa.edu). Pure tones were generated in Matlab, with a sampling rate of 44.1 kHz and 5 msec onset and offset linear ramp. All pure tones and piano tones were 1 sec in length. The specific pure tone and piano tone frequencies are located in appendix 1. 12 Lexical decision task In this task, participants will be asked to determine whether a stimulus was a real word or nonword (e.g., tall and lale). Twenty four real words and twenty four nonwords were selected from the English Lexicon Project (Balota et al. 2007). All real words and nonwords were monosyllabic and consisted of three phonemes, and were four or five letters long in orthography. A complete list of the real words and nonwords can be found in appendix 2. The average durations of the recorded real word stimuli and nonword stimuli were both 0.50 seconds. Sentence judgement task In this task, participants will be asked to determine whether the sentence was expected (e.g., The farmer spent the morning milking his cows.) or unexpected (e.g., The farmer spent the morning milking his cars.) in meaning. Twenty four sentences with expected endings were selected from the Block and Baldwin (2010) sentence completion norms. The sentence-final words of the expected sentences had a 92% or greater expectancies based on the sentence final words generated in the Block and Baldwin norms. The expectancies were based on the responses of 377 participants (18 to 51 years of age) who were given sentence stem that had the final word missing and were instructed to give a single word that would complete each sentence with a likely ending. Twenty-four unexpected final words were also selected, which resulted in 24 additional sentences that did not have an expected ending. The sentences both expected and unexpected 13 endings are listed in appendix 3. The average durations of the expected-ending sentences and unexpected-ending sentences were 3.47 seconds and 3.49 seconds, respectively. All of the speech stimuli (words and sentences) were recorded by the same male, native speaker of American English, and all modifications were made using the Praat software. For the sentence stimuli, at least two versions of the expected ending sentence was recorded. One of these recordings was used to generate the sentence stem--i.e., the sentence without the final word. A different recording of the same sentence was used to splice out the sentence-final word. The sentence-final words (both expected and unexpected) were therefore both created by splicing together the sentence stem and a the sentence-final word from two different recordings. Further, the expected and unexpected sentences contained the exact same recording of the sentence stem. Additionally, in order to avoid any unintended peculiarities in speech production when only sentences are semantically unlikely, 24 additional sentences were created and recorded such that the unexpected sentence-final words were recorded in a semantically likely context. For example, for the expected sentence, ‘He loosened the tie around his neck,’ and the unexpected sentence, ‘He loosened the tie around his freezer,’ the non-stimulus sentence, ‘He placed the pie in his freezer’ was also recorded. The sentence-final word, ‘freezer’ from the non-stimulus sentence (i.e., recorded in a semantically likely context) was spliced onto the sentence stem to create the unexpected stimulus sentence that was presented to the participants.. Each participant heard all 24 tones. The lists of real words, nonwords, expected sentences, and unexpected sentences were split in half, such that each participant heard a list of 12 unique items per stimulus type, and no participant heard the same sentence root more than 14 once. List assignment was counterbalanced across subjects, so that each list was heard across the same number of subjects by the end of the study. Procedure Pupillary Light Reflex Measures Participants adapted to the dimly lit room for at least 10 minutes before any pupillary response recordings began. At the beginning of each block, participants’ pupillary response to light stimuli without auditory stimuli were measured for the calculation of each participant’s baseline pupil diameters. To elicit maximum pupil dilation, a black computer screen was presented for 60 seconds, followed by a white screen for 60 seconds. A total of three pupillary light reflex measures were collected from each participant and averaged for each participant. After the participant completed the pupillary light reflex block, the computer screen remained gray throughout the experimental task as the gray scale elicits an intermediate pupil size that is halfway between the minimum and maximum measure that were calculated (Winn et al. 2015) Figure 1. Measuring the physiological pupillary response. Participants adapted to the room environment for 10 minutes before any recordings. Participants were presented a black screen following by the white screen for one minute each for the purpose of eliciting maximum pupil dilation and maximum pupil constriction. A gray screen is used for the rest of the experimental trials. 15 Participants were presented with three blocks of 24 trials: tones, words, and sentences. Order of blocks were counterbalanced across participants. In each trial, participants were presented with an auditory stimulus, then made a two-alternative, forced-choice response on a keyboard. A fixation cross was presented in the center of the computer screen to keep participants’ gaze fixed throughout each trial. In each trial, participants’ pupil size were recorded from 5 seconds before the stimulus onset and continued until 10 seconds after the stimulus offset, to allow participants’ pupils to return to baseline after each stimulus. Participants will make a two-alternative, forced-choice response 10 seconds after the stimulus offset. Experimental Trials In the experimental trials, participants heard three different types of stimuli: tones, words, and sentences. Participants were asked to focus on a fixation cross throughout each trial for pupil size recordings. The auditory stimulus was presented to the participants at 65 decibels through insert headphones. The stimulus played 5 seconds after trial initiation. 10 seconds after stimulus offset, a cue to respond was visually presented on the computer screen. Figure 2 shows the experimental sequences. For the tone task shown in Figure 2A, participants were asked to indicate with a keypress whether or not the sound was a pure tone or piano tone. For the lexical decision task shown in Figure 2B, participants were asked to indicate whether the word was a real word or not a real word. For the sentence task shown in Figure 2C, participants were asked to indicate whether the sentence was expected or unexpected. Three practice trials were given at the beginning of each block to ensure that the participant understood the upcoming task. Participants’ pupil size was recorded throughout each trial. 16 Pupil size data collection Pupillary response were recorded using the Eyelink 1000 Plus eye tracker (SR Research) at a 1000 Hz monocular high-speed sampling rate. In each trial with an auditory stimulus, change in pupil size was recorded from 5 seconds before the onset of the auditory stimulus (tone, word, or sentence) until 10 seconds after the offset of the auditory stimulus. Pupil size from the onset of the tone, word, or sentence-final word were analyzed across the five parameters during dilation and constriction. For each trial, pupil diameter data were calculated from 20 seconds from the onset of each trial (i.e., 5 sec before stimulus onset). The data were first coded for blinks; a blink was defined as any time pupil diameter data were missing or more than three standard deviations below the mean of that trial. Pupil data were “de-blinked” with linear interpolation between 40 msec before onset of a blink and 128 msec after the end of a blink. This time range was selected to account for the decrease in pupil diameter as the eyelids start to veil the pupil before complete closure, consistent with the time window used by Zekveld et al. (2014). Pupil data were smoothed using a running average across 250 msec before and after each sample. The pupil data from each trial were then adjusted to be centered around the pupil size at stimulus onset to calculate the relative pupil dilation. For the sentence task, pupil data from each trial were also temporally adjusted to be aligned at the onset of the final word of each sentence. 17 2A. Tone Identification Task 2B. Lexical Decision Task 2C. Sentence Judgement Task Figure 2: Procedure of experimental trials. 18 RESULTS I will first describe the general differences across the three tasks. Because each task has two variations, I will then describe the effect of these variations within each task. Across-task differences in pupil size Peak pupil size Figures 3A-C show the mean pupil size over time for the tone identification task, the lexical decision task, and the sentence judgement task prior to the participants giving their button press response. The vertical blue line represents the onset of the stimulus. It can be seen that the presentation of the stimuli causes a sharp increase in adjusted pupil size followed by a decline in the adjusted pupil size. It can be seen in all cases that mean pupil size prior to stimulus onset is larger than the eventual post-stimulus pupil size. This may be due to participant’s anticipation prior to the stimulus onset. Figure 4 shows the mean peak pupil sizes plotted as a bar graph. As can be seen in Figures 4, peak pupil size responded the greatest to lexical decision task, followed by sentence judgement task, and finally tone identification task. Peak pupil size for the lexical decision task (M = 222.3, SD = 164.8) was significantly larger than the peak pupil size for tones identification task (M = 175.7, SD = 122.4), t(47) = 2.49, p < 0.01. This result suggests that listening to and making a judgement on linguistic stimulus elicits a larger processing demand that was reflected on the peak pupil size. Then, we compared the effects of lexical decision task versus sentences judgement task on peak pupil size. In this case, there was no significant difference in mean peak pupil size between lexical decision and sentences judgement task. There was no significant difference in mean peak pupil size between words (M = 222.3, SD = 164.8) and sentences (M = 19 216.2, SD = 154.2), t(47) = 0.32, n.s. However, there was a significant difference between peak pupil size between tones identification task and sentences judgement task, t(47) = 2.03, p < 0.05. Together, these results suggest that making a judgement on linguistic stimuli elicit a greater peak pupil size from listeners than non-linguistic stimuli. Therefore, listening to and making judgements on words and sentences may require a greater processing load, as reflected by greater peak pupil size. Figures 3. Average pupil dilation over time from 5 sec before stimulus onset (indicated with a blue vertical line) until 10 sec after stimulus onset, for all three tasks (A: tones, B: words, C: sentences). 20 Figure 4. Average peak pupil amplitude across all conditions. Blue bars represent piano tones, words, and expected sentences; red bars represent pure tones, nonwords, and unexpected sentences. Within-task differences in pupil size As suggested by visual inspection of Figure 4, there was no significant difference in the mean peak pupil size between identifying piano tones (M = 170.5, SD = 118.9) and identifying pure tones (M = 180.9, SD = 128.1), t(23) = 0.54, n.s.. There was, however, a significant difference on mean peak pupil dilation between responses to words (M = 191.6, SD = 144.7) and nonwords (M = 252.9, SD = 180.5), t(23) = 2.44, p < 0.05. These results suggest that there is a significant effect of lexical decision process on pupil dilation. There was no significant difference between the mean peak size elicited by making a judgement on expected sentences (M = 210.5, SD = 154.6) and unexpected sentences (M = 221.9, SD = 156.9), t(23) = 0.45, n.s. 21 Between-task rate of increase to peak We also compared the effect of the three tasks on the mean rate of increase to peak amplitude. This was calculated as the slope between the pupil sizes at stimulus onset and at peak pupil size. There was a significant difference between the mean rate of increase for the lexical decision task (M = 0.158, SD = 0.123) and the sentences judgement task (M = 0.074, SD = 0.059), t(47) = 5.58, p < 0.0001. In addition, there was a significant difference between the mean rate of increase for the tone identification task (M = 0.139, SD = 0.103) and the sentence judgement task (M = 0.074, SD = 0.059), t(47) = 4.9503, p < 0.0001. There was, however, no significant difference between the tone identification task (M = 0.139, SD = 0.103) and the lexical decision task (M = 0.158, SD = 0.123), t(47) = 1.41, n.s. 22 DISCUSSION Changes in pupil size have been an increasingly common method for measuring perceptual and cognitive effort. In spite of its common use, a comparison of the pupillary response to different types of cognitive tasks within the same experiment have not been extensively studied. The present experiment was conducted to examine the peak amplitude and the rate of increase of the pupillary response evoked by processing auditory stimuli varying in levels of complexity. The levels of complexity included a tone identification task, a lexical decision task, and a sentence judgement task. I hypothesized that, if processing load is reflected by peak pupil amplitude, then amplitude would be greatest in the sentence judgement task compared to the lexical decision task and the tone identification task. I further hypothesized that amplitude would be greater in the lexical decision task than the tone identification task. The results of this study demonstrate that my hypotheses were only partially confirmed. The results of this study confirmed that pupillary responses were sensitive to the difference in auditory decision task. The pupillary changes elicited by listening to and making a judgement on the sentences and the lexical decision task were significantly different to that of tones. Specifically, the peak amplitude was greater for both the lexical decision task and the sentences judgement task compared to the tone identification task. This demonstrated that, if peak amplitude was indicative of processing load, processing a stimulus that contains linguistic components (i.e., words and sentences) was different from processing a stimulus that has no linguistic components (tones). 23 One could argue that the tasks used in this thesis do not impose much cognitive challenge on participants since they only have to listen to and make judgements on the stimuli. The current results revealed that there is a consistent difference in pupil response in judging linguistic versus nonlinguistic stimuli despite the low task difficulty. Therefore, I conclude that the observed task-evoked pupil dilation was indeed sensitive to the differences in the types of auditory tasks and stimuli being perceived. In addition, we also observed that pupils are sensitive to the meaningful-ness of words. I found that pupils dilate greater when hearing a meaningless stimulus (a nonword) compared to hearing a meaningful word (real word). It is possible that the process of deciding that a stimulus is a nonword poses greater effort than deciding that a stimulus is a real word that would be familiar to the listener. That is, to process a nonword would require participants to hold that stimulus in memory, search through their lexicon of words they know, and determine the nonexistence of the nonword in their lexicon. Secondly, the chance of encountering nonwords in real life is minimal. Thus, hearing such low frequent item could initiate a surprisal effect and thus reflect upon their pupil dilation. Overall, this suggests that processing a nonword requires greater effort than recognizing real, familiar words and this increase level of processing was able to be captured through the use of pupillometry. The results revealed that the peak amplitude elicited by making a judgement about the meaningfulness of a sentence was not greater than that of making a lexical decision. There was also no significant difference in peak pupil size when responding to a meaningful sentence or a sentence with an incongruent final word. It is possible that the perception and processing of 24 sentences is facilitated by their syntactic structure, which made sentence processing as easy as processing a word without context. The results revealed that the mean rate of increase to peak elicited by lexical decision task was greater than sentence judgement task. Furthermore, the mean rate of increase elicited by tone identification task was greater than sentence judgement task. Although significantly different, the mean rate of increase to peak for sentence judgement task is dictated by the latency to peak. Therefore, I believe that the observed differences are due to the nature of the sentence stimuli being much longer than tones and words. As will be indicated, pupil size measurements have been used in cognitive task comparing younger and older adults. An important question is whether the kinetics of older adults evoked pupillary response differs from that of younger adults. In the next chapter, I present a pilot study with older adults to indicate the direction of possible future research. 25 Chapter 2 Effects of healthy aging on pupillary kinetics in response to auditory stimuli that vary in complexity: A pilot study INTRODUCTION Despite the wide use of pupillometry as a method to index cognitive load, cognitive-related pupillometry raises special issues in older adults. Most notably, older adults exhibit senile miosis, which is an age-related condition where pupillary response is more limited in range of dilation. As a result, pupillary response may not accurately index the extent of cognitive effort exerted by older adults relative to younger adults in cognitive tasks due to the influence of age related autonomic nervous system changes on pupil dilation. Studies have used pupillary response to suggest that older adults expend greater cognitive effort during spoken language comprehension than young adults (Piquado et al. 2010, Kramer et al. 2007, Zekveld et al. 2013, 2014). However, there is a deeper question that has not been addressed in the literature. As will be noted later, a common adjustment for senile miosis is to use the ratio of the change in the pupil size relative to the maximum excursion the participant shows to the light reflex. This method implicitly assumes that the kinetics of the pupillary response are the same for young and older adults. This relates to potential age related differences in the dynamics of the pupillary response in older versus young adults. 26 Effect of Aging on Pupillary Light Reflexes There are normal, age-related changes to autonomic nervous system functioning, which dictates the pupillary light reflex through the Locus Coeruleus-Norepinephrine control system (LC-NE) (Wilhelm, Wilhelm, & Ludtke, 1999, Alnaes et al. 2014). The locus coeruleus, located in the rostral pons in the brainstem, contains norepinephrinergic neurons that project widely across areas of forebrain and cerebellum. In response to stress, the LC-NE system alters the balance of activity between the sympathetic and the parasympathetic nervous system, which dictates the phasic changes of pupils. On the basis of previous literature, the PNS activity is reduced in healthy aging which is necessary for the innervation of pupil dilation. In order to evaluate the pupil’s sensitivity to PNS activity regardless of age-related changes in the autonomic nervous system function, there is a need to further examine specific pupil parameters to fully understand the relationship between aging and pupil light reflex. Further studies have compared pupillary kinetics of younger adults to that of older adults with various manipulations (Bitsios et al. 1996, Pozzessere et al. 1996). Bitsios et al. (1996) illustrated the differences between young and older adults’ pupillary response to incoming light. They identified that reduced pupil size, diminished darkness reflex amplitude and velocity, and prolonged recovery time of the light reflex were consistent with the SNS deficit and or the PNS disinhibition with older age. They hypothesized that age-related pupillary movement abnormalities may occur due to a presumed reduction of SNS tone in older adults. Such change in kinetic components of the pupillary response indicate that healthy aging alters the balance of the SNS and the PNS activity and is shown by the difference in the pupillary response to ambient light. In addition, Pozzessere et al. (1996) observed that pupillary constriction amplitude is 27 smaller and the constriction velocity is slower in older adults than that of younger adults’ PLR. These observations of age-related changes in the pupillary kinetics shows a linear reduction with increasing age, which suggest that as age increases, older adults’ pupils will take a longer time to constrict and will not constrict greatly. These previous studies have laid the groundwork for future research on how healthy aging alters the mechanism of autonomic nervous system regulation on pupillary response. However, the influence of age-related autonomic nervous system changes on pupillary response poses a research problem for scientists who utilize pupillometry to index cognitive load in older adults. Such change includes the SNS tone deficit and the PNS tone disinhibition would potentially compromise pupillometry as a comparative cognitive load assay in older adults since the pupillary parameters might systematically underestimate older adult’s degree of cognitive load. Thus, the current pilot study examines pupil peak amplitude, maximum velocity to peak, and latency to peak. These pupillary parameters were measured in prior literature and indicative of autonomic nervous system activities (Bitsios et al. 1996, Pozzessere et al. 1996, Muppidi et al., 2013). In conclusion, the physiology of pupillary activities can be altered by healthy aging which may potentially produce age differences in the kinetics of pupillary response to cognitive effort. The consequences of this change could be: the smaller mean pupil size at rest, the range of excursion from minimum to maximum pupil size, and importantly, the dynamics of the pupil size movement within this range. 28 Effects of aging on pupillometry in response to cognitive efforts The use of pupillometry as a measure of cognitive load in older adults in comparison to younger adults has been complicated by aging. To the best of our knowledge, aging results in the deficits of autonomic nervous system which cause a smaller pupil size and restricted range in pupil dilation in older adults. Because of these unknown mechanisms and resulting complications, pupil dilation elicited by cognitive tasks in older adults might not be as responsive as that of younger adults which would not accurately index the amount of cognitive resources expended during tasks. For instances, Kramer, Kapteyn, Festen, and Kuik (1997) applied the method of pupillometry to the listening of sentences in noises by normal-hearing and hearing-impaired listeners. They demonstrated that decreased in signal-to-noise ratio results in progressive increase pupil dilation with normal hearing, with this effect being differentially larger for hearing impaired listeners. The hearing-impaired group was older (mean age: 44) than the normal-hearing group (mean age: 29). Winn et al. (2016) examined the impact of context on speech intelligibility and ongoing listening effort (reflected by the change in pupil dilation over time) among participants who have normal hearing (NH) and are cochlear implant (CI) users. They revealed that words preceded by relevant semantic context elicits a reduction in pupil dilation in listening effort. In addition, the CI users elicited smaller pupil dilation compared to NH due to their existing experience with distorted resolution. However, the potential effects of age was noted in the study as their CI participants had more older individuals (mean age: 58.52) who showed generally a smaller pupil dilation in response than their younger participants. Therefore, the degree of pupillary dilation 29 becomes ambiguous as the pupillary change can respond to age-related autonomic nervous system change and, or cognitive expenditure during tasks. Research Motivation My research motivation arises from the notion that older adults have a physiological change in the pupillary response during cognitive tasks. To the best of our knowledge, changes to pupillary light reflex due to autonomic nervous system changes are well documented. Pupillary response due to cognitive effort has also been well established, as has the association between aging and increased cognitive effort during spoken language processing. However, the effects of aging on pupillary response due to cognitive effort has received little to none attention. As indicated in my literature review, task-evoked pupillary response has been used as a measure of cognitive effort for over 50 years and more recently, as a way of comparing the relative effort required by young versus older adults to accomplish the same cognitive task. It can be seen that the presence of senile miosis demands that one cannot use raw pupil size for comparative measures. The most common method of adjustment is to use the ratio of the change in the pupil size relative to the maximum excursion the participant shows to the light reflex. This simple adjustment implicitly assumes that the kinetics of the pupillary response are the same for young and older adults. This presumption has not been tested in the context of pupillometry as a measure of cognitive effort. The purpose of this pilot study is to close this gap in our knowledge. 30 Hypotheses We expect the effects of aging to underlie these mechanisms will contribute to different elicitation of pupillary kinetics between older and younger adults in addition to cognitive task complexity. Based on previous literature, we hypothesized that healthy aging will reflect on the peak amplitude (range of excursion) and mean rate of increase to peak. Specifically, we hypothesized that mean rate of increase to peak and peak amplitude of older adults are slower and smaller than that of younger adults, respectively. Preliminary Results and Discussion Four older adults participated in this pilot study (age: M = 70.2, SD = 3.3; all female). Their verbal ability was assessed by the Shipley Vocabulary Test, in which these older adults scored similarly to the younger adults (young: M = 14.1, SD = 1.4; older: M = 14.5, SD = 2.9). The older adults had relatively good (age-normal) hearing, ranging from 18.3 to 30 dB pure tone average across 500, 1000, 2000, and 4000 Hz. Preliminary data revealed that the peak pupil amplitude across all conditions are slightly smaller in older adults compared to younger adults. Figure 6 shows the average pupil dilation for tone identification task between younger and older adults. Consistent with previous literature, we are observing that peak amplitude are generally smaller in older adults in the pilot study (though these differences appear quite small). In addition, as elaborated on Figure 10, we also observed that the rate of increase to peak is notably less steep in older adults than younger adults across almost every condition. We also examined the latency to peak to investigate the timing needed for older adults’ pupil responses to respond to the auditory stimuli. So far, as shown in figure 9, 31 latency to peak shows more sensitivity to judging sentence than to our manipulations of words and tones. Overall, based on the current trends, this preliminary data suggests that healthy aging alters the sensitivity of the pupils. If significant, this could reflect the extra exertion of effort older adults need on language processing. Therefore, these preliminary trends suggest that aging could be an important variable to consider when utilizing pupillometry as a method to index effort across differently aged populations. The interpretation of older adults’ pupillary response requires careful consideration as less excursion and slower kinetics are observed. Figure 5. Average pupil dilation over time from 5 sec before stimulus onset (indicated with a blue vertical line) until 10 sec after stimulus onset, for tone identification task. The left graph represents the mean peak amplitude elicited by younger adults (n = 24). The right graph represents the mean peak amplitude elicited by older adults (n = 4). 32 Figure 6. Average pupil dilation over time from 5 sec before stimulus onset (indicated with a blue vertical line) until 10 sec after stimulus onset, for lexical decision task. The left graph represents the mean peak amplitude elicited by younger adults (n = 24). The right graph represents the mean peak amplitude elicited by older adults (n = 4). 33 Figure 7. Average pupil dilation over time from 5 sec before stimulus onset (indicated with a blue vertical line) until 10 sec after stimulus onset, for sentence judgement task. The left graph represents the mean peak amplitude elicited by younger adults (n = 24). The right graph represents the mean peak amplitude elicited by older adults (n = 4). Figure 8. Average peak pupil amplitude across all conditions. Younger adults’ (YA) data are the darker bars on the left side within each task and older adults’ (OA) data are the lighter bars on the right. Blue bars represent piano tones, words, and expected sentences; red bars represent pure tones, nonwords, and unexpected sentences. 34 Figure 9. Latency to peak across all conditions. Younger adults’ (YA) data are the darker bars on the left side within each task and older adults’ (OA) data are the lighter bars on the right. Blue bars represent piano tones, words, and expected sentences; red bars represent pure tones, nonwords, and unexpected sentences. Figure 10. Rate of increase of pupil size across all conditions. Younger adults’ (YA) data are the darker bars on the left side within each task and older adults’ (OA) data are the lighter bars on the right. Blue bars represent piano tones, words, and expected sentences; red bars represent pure tones, nonwords, and unexpected sentences. 35 CONCLUSIONS The current findings demonstrate that the task-evoked pupillary response to auditory tasks is a sensitive indirect measure of cognitive processing load. Processing a stimulus that contains a linguistic component (i.e., words and sentences) requires greater processing demands than just identifying a stimulus with no linguistic component (tones). Deciding whether a stimulus is a nonword requires greater processing demands than deciding whether a stimulus is a real word. Thus, the task-evoked pupillary response reflects the differences in task demands and stimulus characteristics in auditory decision tasks. Preliminary data suggests that older adults performing the same auditory decision tasks as young adults revealed a smaller change in pupil size and a slower mean rate of increase to peak. These results suggest that there is a need for caution when making comparisons of relative cognitive effort based on pupillometry. ACKNOWLEDGEMENTS I would like to express my deepest gratitude to Dr. Arthur Wingfield, Dr. Eriko Atagi, Dr. Maxim Bushmakin, and the members of Memory & Cognition Laboratory of Brandeis University for their support, assistance, and encouragement throughout this research and writing process. 36 APPENDICES Appendix 1 Tone (Piano, Pure) Frequency (Hz) C 261.6 C♯ 277.2 D 293.7 E♭ 311.1 E 329.6 F 349.2 F♯ 370.0 G 392.0 G♯ 415.3 A 440.0 B♭ 466.2 B 493.9 37 Appendix 2 Words Words (IPA transcription) Nonwords Nonwords (IPA transcription) tall tɔl lale leɪl share ʃɛr wone woʊn rear rɪr dake deɪk pine pajn fane feɪn pack pæk lome loʊm mash mæʃ gake ɡeɪk mass mæs hars hɑrz mall mɔl hake heɪk deal dil loke loʊk seed sid pame peɪm seat sit pake peɪk mill mɪl mide maɪd gate ɡet rame reɪm fill fɪl reat rit fate fet hame heɪm core kɔr leal lil came kem sone soʊn bill bɪl cose koʊs tear tɛr lide laɪd mate met rone roʊn tack tæk saze seɪz make mek lave leɪv fine fajn poot put dear dɪr mape meɪp 38 Appendix 3 Sentence Root Expected Ending Unexpected Ending He loosened the tie around his neck freezer The girl went to the library to borrow a book shoe Father carved the turkey with a knife sock The blue pen quickly ran out of ink energy After hitting the iceberg, the ship began to sink fly They sat together without speaking a single word hat He went to the salon to color his hair nose The man proposed and gave her a diamond ring cheese The boy blew out the candles on his birthday cake fish She could tell he was mad by the tone of his voice coffee The package was sent through the mail bottle He went to the bakery to buy a loaf of bread chicken The farmer spent the morning milking his cows car After the argument the girl went to her room and slammed the door sky When the alarm rang the firefighter slid down the pole crown She didn’t have her watch so she asked for the time pig In the shower he washed his skin with soap birds He mailed the letter without a stamp chair To hang the picture Ted needed a hammer and nail turkey For his date Tom bought a long stemmed rose coat The groom promised to keep his vows thumb Finished in 15 minutes, he thought the exam was a piece of cake balloon The dolphins are swimming in the ocean ground The photographer used her camera to take a photo break 39 Bibliography Alnæs, D., Sneve, M.H., Espeseth, T., Endestad, T., van de Pavert, S.H.P., Laeng, B., 2014. Pupil size signals mental effort deployed during multiple object tracking and predicts brain activity in the dorsal attention network and the locus coeruleus. J. Vis. 14 (4), 1. h Balota, D.A., Yap, M.J., Cortese, M.J., Hutchison, K.A., Kessler, B., Loftis, B., Neely, J.H., Nelson, D.L., Simpson, G.B., & Treiman, R. (2007). The English Lexicon Project. Behavior Research Methods, 39, 445-459. Beatty, J. (1982). Task-evoked pupillary responses, processing load, and the structure of processing resources. Psychological Bull, 91, 276292. Beatty, J., & Lucero-Wagoner, B. (2000). The pupillary system. In: J. T. Cacioppo, L. G. Tassinary, & G. G. Berntson (Eds.), Handbook of psychophysiology (2nd ed., pp. 142162). New York, NY: Cambridge University Press Beatty, J & Kahneman, D. Pupillary changes in two memory tasks. Psychonomic Science, 5(10):371–372, 1966 Bergamin O, Schoetzau A, Sugimoto K, Zulauf M (1998) The influence of iris color on the pupillary light reflex. Graefes Arch Clin Exp Ophthalmol 236: 567–570. pmid:9717650 Bitsios, P., Prettyman, R., & Szabadi, E. (1996). Changes in autonomic function with age: a study of pupillary kinetics in healthy young and old people. Age and Ageing, 25(6), 432-438. Block, C. K., & Baldwin, C. L. (2010). Cloze probability and completion norms for 498 sentences: Behavioral and neural validation using event-related potentials. Behavior Research Methods, 42(3), 665-670. Granholm E., Steinhauer S. R. (2004). Pupillometric measures of cognitive and emotional processes. International Journal of Psychophysiology, 52 (1), 1–6, doi:10.1016/j.ijpsycho.2003.12.001. Hess, E. and Polt, J,. Pupil size in relation to mental activity during simple Problem-Solving. Science, 143(3611):1190–1192, March 1964. doi: 10.1126/science.143.3611.1190. Hoeks, B. (1995). The pupillary response as a measure of processing load (PhD thesis). Katholieke Universiteit Nijmegen, The Netherlands. Just, M. A., & Carpenter, P. A. (1993). The intensity dimension of thought: Pupillometric indices of sentence comprehension. Canadian Journal of Experimental Psychology, 47, 310339. Kahneman, D., & Beatty, J. (1966). Pupil diameter and load on memory. Science, 154(3756), 1583-1585. Kramer, S. E., Kapteyn, T. S., Festen, J. M., & Kuik, D. J. (1997). Assessing aspect of hearing handicap by means of pupil dilation. Audiology, 36, 155164. Kuchinsky, S. E., Ahlstrom, J. B., Vaden, K. I., Cute, S. L., Humes, L. E., Dubno, J. R., & Eckert, M. A. (2013). Pupil size varies with word listening and response selection difficulty in older adults with hearing loss. Psychophysiology, 50(1), 23-34. Loewenfeld IE, Lowenstein O (1999) The Pupil: Anatomy, Physiology, and Clinical Applications: Butterworth-Heinemann. 40 Mazunik, Evan. "University of Iowa Electronic Music Studios." University of Iowa Electronic Music Studios. N.p., Nov.-Dec. 2001. Web. 01 Feb. 2017. Muppidi, S., Adams-Huet, B., Tajzoy, E., Scribner, M., Blazek, P., Spaeth, E. B., ... & Vernino, S. (2013). Dynamic pupillometry as an autonomic testing tool. Clinical Autonomic Research, 23(6), 297-303. Naber M, Alvarez GA, Nakayama K. Tracking the allocation of attention using human pupillary oscillations. Frontiers in Psychology. 2013;4:919. doi:10.3389/fpsyg.2013.00919. Piquado, T., Isaacowitz, D., & Wingfield, A. (2010). Pupillometry as a measure of cognitive effort in younger and older adults. Psychophysiology, 47(3), 560-569. Pozzessere G, Valle E, Rossi P, Petrucci B, Ambrosini A, D'Alessio M, et al. (1996) Pupillometric evaluation and analysis of light reflex in healthy subjects as a tool to study autonomic nervous system changes with aging. Aging (Milano) 8: 55–60. Tun, P. A., McCoy, S., & Wingfield, A. (2009). Aging, hearing acuity, and the attentional costs of effortful listening. Psychology and Aging, 24(3), 761-766. Van Gerven, P. W., Paas, F., Van Merriënboer, J. J., & Schmidt, H. G. (2004). Memory load and the cognitive pupillary response in aging. Psychophysiology, 41(2), 167-174. Verney, S. P., Granholm, E., & Marshall, S. P. (2004). Pupillary responses on the visual backward masking task reflect general cognitive ability. International Journal of Psychophysiology, 52(1), 23-36. Wilhelm B., Wilhelm H., Lüdtke H. (1999). Pupillography: principles and applications in basic and clinical research, in Pupillography: Principles, Methods and Applications, eds Kuhlmann J., Böttcher M., editors. (München: Zuckschwerdt Verlag; ), 1–11 Wingfield, A., Amichetti, N. M., & Lash, A. (2015). Cognitive aging and hearing acuity: Modeling spoken language comprehension. Frontiers in Psychology. 6:684. doi: 10.3389/fpsyg.2015.00684 Winn, Matthew B., Jan R. Edwards, and Ruth Y. Litovsky. “The Impact of Auditory Spectral Resolution on Listening Effort Revealed by Pupil Dilation.” Ear and hearing 36.4 (2015). Zekveld, A. A., Kramer, S. E., & Festen, J. M. (2010). Pupil response as an indication of effortful listening: The influence of sentence intelligibility. Ear Hear, 31(4), 480490. Zekveld, A. A., Kramer, S. E., & Festen, J. M. (2011). Cognitive load during speech perception in noise: The influence of age, hearing loss, and cognition on the pupil response. Ear and Hearing, 32(4), 498-510. 41
© Copyright 2025 Paperzz