VISUAL COGNITION, 0000, 00 (00), 000 /000 Explicit memory for rejected distractors during visual search Melissa R. Beck and Matthew S. Peterson Psychology Department, George Mason University, Fairfax, Virginia, USA Walter R. Boot Beckman Institute and Department of Psychology, University of Illinois Urbana-Champaign, Champaign, Illinois, USA Miroslava Vomela Psychology Department, George Mason University, Fairfax, Virginia, USA Arthur F. Kramer Beckman Institute and Department of Psychology, University of Illinois Urbana-Champaign, Champaign, Illinois, USA Although memory for the identities of examined items is not used to guide visual search, identity memory may be acquired during visual search. In all experiments reported here, search was occasionally terminated and a memory test was presented for the identity of a previously examined item. Participants demonstrated memory for the locations of the examined items by avoiding revisits to these items and memory performance for the items’ identities was above chance but lower than expected based on performance in intentional memory tests. Memory performance improved when the foil was not from the search set, suggesting that explicit identity memory is not bound to memory for location. Providing context information during test improved memory for the most recently examined item. Memory for the identities of previously examined items was best when the most recently examined item was tested, contextual information was provided, and location memory was not required. Please address all correspondence to Melissa R. Beck, Department of Psychology, George Mason University, MS 3F5, 4400 University Dr., Fairfax, VA 22030, USA. E-mail: [email protected] This work was supported by grants from the National Institute of Health to MSP (R01 MH64505) and the Office of Naval Research to AFK and WRB. Parts of this research were presented at the 2005 Vision Sciences Society in Sarasota Florida. http://www.psypress.com/viscog # 2006 Psychology Press Ltd DOI: 10.1080/13506280600574487 2 BECK ET AL. It is generally agreed that memory is involved in visual search, but that it’s role is limited (for review see Wolfe, 2003). This limited capacity role of memory in visual search has been demonstrated by the tendency for fixations and attention to be biased away from previously examined items (Klein & MacInnes, 1999; Kristjansson, 2000; McCarley, Wang, Kramer, Irwin, & Peterson, 2003; Peterson, Kramer, Wang, Irwin, & McCarley, 2001; Takeda, 2004; but see Horowitz & Wolfe, 1998). This memory allows people to avoid needlessly reinspecting items that have already been identified as not being the target of search. The form of memory used to bias attention away from previously examined items is largely unknown (but see Boot, McCarley, Kramer, & Peterson, 2004). It has been demonstrated that changing the location but not the identity of a previously examined item disrupts the ability to bias attention away from the item (Beck, Peterson, & Vomela, in press). Consequently, it would appear that attention in visual search is guided by memory for the locations of previously inspected items rather than their identities. Although memory for examined item’s identities may not be used to guide attention, here we examine the extent to which explicit memory for the identity of rejected distractors is acquired during visual search. MEMORY IN VISUAL SEARCH Researchers supporting the role of memory in visual search argue for the necessity of keeping track of previously examined items in order to prevent unnecessary reinspection of these items (Dodd, Castel, & Pratt, 2003; Klein & MacInnes, 1999; Müller & von Muhlenen, 2000; Takeda & Yagi, 2000). Peterson et al. (2001) demonstrated that the probability of revisiting an item during search is less than chance for up to 11 or more previously examined items. One possible explanation for the lack of revisits observed by Peterson and colleagues is that some form of prospective memory or systematic scanpath planning was used to bias attention towards unexamined items rather than away from examined items. For example, a simple strategy of searching from top to bottom would prevent revisits even in the absence of some form of retrospective memory for examined items. To prevent scan-path strategies from contaminating memory estimates, McCarley and colleagues (2003) used an eye movement contingent display. In their task, no more than three items were visible at any one time: The currently fixated item and two potential saccade targets. Items were the same size and the same spacing as used by Peterson et al. The key was that during an eye movement, two new potential saccade targets appeared and the remaining items were removed. This forced participants to choose which of two items to examine. After IDENTITY MEMORY DURING SEARCH 3 several saccades, one of the potential saccade targets was always an unexamined item (new item) and the other was an item that had been examined previously (old item) in the trial. If there is memory, then the new items should always be chosen, and if there is no memory, then the new item will be chosen 50% of the time. McCarley and colleagues found that participants moved their eyes to the new item more frequently than predicted by chance, and this effect lasted for roughly the last four to five items examined. This suggests that people have retrospective memory for the last four examined items or locations. Retrospective memory for the last four to five items is driven by the locations of these items rather than their identities. For example, Beck et al. (in press) demonstrated that changing the location of an examined item disrupted search, whereas changing the identity of an examined item did not. In their task, search items (rotated Ts and Ls) were surrounded by large unique coloured shapes (e.g., a red circle). While participants performed the visual search task, the shape surrounding a previously examined item changed identity (e.g., red circle changed into a yellow square) or a previously examined item moved to a new location. When a previously examined item changed identity, fixations were still biased away from the item. However, when a previously examined item moved from one location to another, fixations were no longer biased away. This suggests that the memory used to bias attention away from previously examined items is based on the items’ locations rather than their identities. One key question remains: Although individuals show memory for the locations of the last few items, why does changing the identity of an examined item have no effect on search? One possibility is that the identity information is remembered, but this information is not used to prevent revisitations. Alternatively, the locations but not the identities of examined items might be remembered, and hence changing the identity of an examined item has no effect on search. Although research examining eye movements during visual search suggests that memory is limited to the location of items and not their identities, there have been several demonstrations of memory for visual details of previous viewed stimuli (Hollingworth & Henderson, 2002; Standing, 1973). Similarly, explicit long-term memory (LTM) tests show memory for specific visual details of distractors during search (Castelhano & Henderson, 2005; Williams, Henderson, & Zacks, 2005). Williams et al. (2005) presented participants with an incidental memory test for the targets and the distractors from a visual search task that was completed at least 10 minutes prior to the memory test. Each distractor and target was viewed on two separate visual search trials, and then memory was tested using a twoalternative forced choice (2AFC) test that required discriminating the target or distractor from a nonsearch item with the same verbal label (e.g., a 4 BECK ET AL. yellow telephone) but different visual features. Memory performance for targets was fairly high (approximately 85%) while performance for distractors was much lower but still above chance (approximately 60%). Castelhano and Henderson (2005) found similar results when participants searched real-world scenes for a specific target and then were asked to discriminate a nontarget object from another object of the same basic level category or from the mirror image of the same object. Performance on the incidental LTM memory test was lower than on an intentional LTM memory test, but still significantly above chance (60 /75%). This suggests that even without the intent to remember, some visual details are stored in LTM. Above chance performance on LTM tests for distractors appears in conflict with the lack of memory disruption when an item changes identity during search. Possibly, distractor identity information is remembered during search, but it is not used to guide visual attention during search. Therefore, an explicit test of memory for the identity of examined distractors during the visual search task may reveal memory for distractor identities. When participants are asked to complete an intentional memory task that does not coincide with a visual search task, they can remember roughly ‘‘5.4 objects worth of information’’ (Irwin & Zelinsky, 2002). In Irwin and Zelinsky’s study, participants were permitted to examine seven objects for 1 / 15 fixations. Then, they were asked to indicate which object was in a particular location. Memory performance for the last three items fixated and the item about to be fixated on the saccade when the memory test was presented was high (/80%). Therefore, viewers have the ability to intentionally encode the identities and locations of recently viewed items. The current studies examine the extent to which observers use this ability during visual search. MEMORY FOR IDENTITY VERSUS LOCATION Previous research suggests that memory for location may have special importance over memory for identity. For example, research suggests an overlap between visual search and spatial, but not object, working memory (Oh & Kim, 2004; Woodman & Luck, 2000). In addition, memory for the locations of distractors across visual search trials can be used to improve the efficiency of visual search (contextual cueing; Chun & Jiang, 1998). Therefore, memory for the locations of distractors during visual search may be more important than memory for the identities of distractors. Theories of how visual information is combined across eye movements generally assume that spatial information is the pointer to a representation, IDENTITY MEMORY DURING SEARCH 5 which contains identity or feature information about previously attended items (Henderson, 1994; Henderson & Anes, 1994; Henderson & Siefert, 2001; Kahneman, Treisman, & Gibbs, 1992). These theories suggest that identity information about an object is tied to spatiotemporal information that is determined by the relative position of the item in relation to other items in the display. If identity and location are stored jointly, memory performance on a test that required only identity information should be equivalent to a test that required identity and location information. However, it has been suggested that features can be stored individually and that binding these features requires maintenance by attention (Wheeler & Treisman, 2002; Wolfe, Oliva, Butcher, & Arsenio, 2002). Therefore, performing a visual search task may use attentional resources that are otherwise used to maintain bound features. This would predict that we should find good memory for individual features but not for these features when referencing them through a particular location. Here we examined the extent to which explicit memory for previously examined distractors is available during visual search. Participants searched displays for one of two possible targets. Occasionally during the visual search task, search was stopped, and memory for one of the previously examined items in the display was tested using a 2AFC. Memory was probed by asking participants which of two objects was in a particular location. We tested memory for the identities of previously examined objects and for these identities bound to specific locations by manipulating the type of foil used in the memory probe. If the foil was another previously viewed distractor, participants needed to remember which item was in a specific location in order to successfully perform the memory task. If the foil was not one of the previously viewed distractors, participants could simply remember which object was viewed during the trial irrespective of location. These experiments allow us to examine the extent to which explicit memory for distractor identity is available during a visual search task and whether memory for identity is bound to location. In addition to examining explicit memory for distractor identity, we will replicate findings showing attentional guidance by memory for distractor locations. Attentional guidance by memory for the locations of previously examined distractors is demonstrated by infrequent (lower than chance) revisits to previously examined distractors. EXPERIMENT 1 The search task in Experiment 1 was similar to the oculomotor contingent paradigm used by McCarley and colleagues (2003), in which only two items from the search set (other than the fixated item) are visible at any one time (see Figure 1). This oculomotor contingent search task was used in order to 6 BECK ET AL. 1 1 2 1 3 Event 1 Event 2 1 2 2 Event 3 4 3 Event 4 4 5 Event 5 Figure 1. The sequence of events within a search trial in Experiment 1. The dashed circle indicates the observer’s point of fixation. Numbers are used to indicate different stimulus items. Stimuli in the actual displays were letters of the alphabet too small to be discriminated without foveation. Note that from Event 3 onward, the observer was forced to choose between executing a saccade from the currently fixated item toward a new item and executing a saccade toward a decoy item that had already been seen. See the text for additional details. Reproduced from McCarley et al. (2003). replicate the finding that there is memory for the locations of the last three to four examined items even when the ability to plan future saccades is eliminated and to ensure that the memory task does not interfere or change search behaviour. Furthermore, we added a memory probe for the last four examined items on a subset of the trials to determine the extent to which there was also explicit memory for the identity of previously examined items. The target was an S or a U, and distractors were most of the remaining letters of the alphabet. On approximately 30% of the trials, the search was stopped, the items were removed from the screen, and a red box appeared around an empty location. Subjects were given a 2AFC task in which one of the items was the item that had appeared at that location and the alternative choice (foil) was an item that had appeared somewhere else (identity bound to location condition) or an item that had not appeared at all (identity only condition). High accuracy in both conditions would indicate that identity information is retained and it is bound to the location of the examined distractor. However, it may be that identity information is retained but not bound to location in which case we would expect high accuracy in the identity condition but not the identity bound to location condition. Finally, if there is no explicit memory for identity we would expect poor memory performance in both conditions. Method Participants. Twenty participants from the University of Illinois were paid US$24 for their participation in three 45-minute experimental sessions on separate days. They were each assigned to either the identity (n/10) or the identity bound to location condition (n/10). The average age of the participants in the identity condition was 23 and the average age of the participants in the identity bound to location condition was 26. All participants demonstrated normal or corrected-to-normal visual acuity. IDENTITY MEMORY DURING SEARCH 7 Apparatus. Stimulus presentation was controlled by a Pentium-based computer. Stimuli were presented on a 21-inch colour SVGA monitor with a resolution of 1280 /1024 pixels and a refresh rate of 85 Hz. Eye movements were recorded using an Eyelink II eyetracker (500 Hz temporal resolution, 0.28 spatial resolution). An eye movement was classified as a saccade if its distance exceeded 0.28 and either its acceleration reached 9500 deg/s2 or its velocity reached 30 deg/s. A chinrest 71 cm from the monitor was used to stabilize head position and the monitor subtended approximately a 27.848 visual angle. Stimuli and procedure. Participants viewed dynamic search displays containing small white letters (0.38 /0.38) on a black background. These letters were too small to be identified without foveation. Participants were asked to search for an S or a U target among distractor letters comprising of the remaining letters of the alphabet from A to V. Participants were asked to press one of two buttons depending on the identity of the target. Only one, two, or three of these items were presented on the screen at any one time. The trial started with participants fixating a white fixation point at the centre of the screen and pressing the spacebar. The fixation point was then removed and the first search item appeared in the display (Event 1; see Figure 1). Participants had to saccade to this item to determine its identity. As participants moved their eyes to this item, a second item was added to the display (Event 2). If the first item fixated was the search target participants had to indicate whether it was an S or a U. If this item was not the target, participants were required to move their eyes to the newly appearing item and determine whether it was the target. During this eye movement another item was added to the display (Event 3). Note that at this point if the fixated item was not the target, participants had the option of two saccade targets with the next eye movement. Participants could either saccade back to the first item visited (old item), or saccade to the uninspected item (new item). These two saccade targets were equidistant from the fixated item. With each additional saccade participants made, one old item and one new item were presented. The old item was sampled randomly from the list of items already presented during the trial. If the randomly chosen item was the item just previously fixated, this item remained on screen. If the randomly chosen item was not the item just previously fixated, the just previously fixated item was removed from the display and the randomly chosen old item was presented in the same location and with the same identity as it had previously. Stimuli were constrained such that the two items presented with each saccade were at least 5.4 degrees from one another and from the about to be fixated item. If no stimuli could be found to satisfy these conditions, the trial terminated. Over the series of events within a trial, participants were 8 BECK ET AL. presented with the choice to saccade to an old item or to saccade to a new item. The old item could be the item fixated one (lag/1) or more fixations ago. Attentional guidance by memory for previously inspected locations is demonstrated in this paradigm by greater than chance selection of the new item, since participants have the opportunity to saccade to either an old item or a new item. Targets occurred on 25% of nonprobe trials. On most trials (67%), the trial continued until the target was found or 11 saccades were made. On the remaining trials (33%), the screen was erased during the saccade following the sixth fixation on an item, and a red box (0.8 /0.8) appeared around one location. Within this box, a question mark appeared, and one letter appeared to the left of this box and one letter appeared to the right of this box. One letter was always the letter that appeared at that location, and this letter could occur equally often to the left or right of the probe box. The letter opposite the letter that was at that location was a foil letter. In the identity only condition, the foil was an item that never appeared during the trial. In the identity bound to location condition, the foil was a letter that participants had fixated during the trial, but occurred at a location other than the probe location. The foil was randomly chosen from the list of old items. In each condition, the probed letter could be the letter fixated one, two, three, or four fixations back (lag). Participants were asked to choose which item was at the probed location by pressing one of two buttons on the controller. Participants were instructed that the search task was the primary task and that the majority of trials would be search trials. Participants completed three blocks of 240 trials. Each block was completed on separate days. Results As was done in McCarley et al. (2003), the first block of trials was considered practice and was not included in the reported analysis because we wanted to be sure that participants were familiar with the task. Excluding this first block did not change the results in any meaningful way. Search performance. If participants failed to respond prior to the 11th fixation on an item, the trial was ended. Participants could fail to respond because no target was presented (after 11 fixations) or because they saw the target but failed to report its presence. Participants failed to respond to the target on 20% of the target present trials in the identity only condition and IDENTITY MEMORY DURING SEARCH 9 0.8 Identity only Proportion of revisits 0.7 Identity bound to location 0.6 0.5 0.4 0.3 0.2 0.1 0 0 1 2 3 4 5 Lag Figure 2. Proportion of revisits to old items in Experiment 1. Chance is .5 (dotted line) because there was always one new item and one old item to choose from. 18% of the target present trials in the identity bound to location condition.1 When participants did respond during a trial, they were 95% accurate in the identity only condition and 97% accurate in the identity bound to location condition. Eye movements. Figure 2 presents the mean proportion of fixations when the old item was chosen as the saccade target over the new item for the nonprobe trials in the identity only and the identity bound to location conditions. Data are plotted as a function of the number of gazes since the old item was last examined (revisit lag). Because there were always two items to choose from (one old item and one new item), the probability of randomly choosing an old item was always .50. One-sample t-tests revealed that old 1 Particularly high miss rates in this paradigm may be attributed to the dynamic nature of the search display or the low probability of a target occurring. Because the item that was just examined would often be removed from the screen after the eyes moved to a new item, participants were often unable to reexamine an item just after it was examined. Therefore, if participants fixated the target without responding, they rarely had a second chance to view the target. This is unlike traditional search paradigms in which participants might fixate the target, move their eyes without recognizing it, and move their eyes back to the target location later during the trial. As noted by Peterson et al. (2001), refixations during a static search display were often directed toward the target after the eyes had fixated too briefly the first time. Another potential explanation might be the low incidence of target occurrence in this paradigm (only 25% of the nonprobe trials). This may have caused participants to adopt a stringent criterion for detecting the target. 10 BECK ET AL. items examined one, two, or three fixations prior to the current fixation were reexamined at rates lower than .50, smallest t(9) / /4.44 at revisit lag 3 for the identity only condition, all p sB/.002, for both the identity only and the identity bound to location conditions. Therefore, there was memory for item location for the past three items examined. Furthermore, there were no differences in the rates of revisits between the identity and identity bound conditions for any of the revisit lags. Explicit memory performance. One-sample t -tests revealed that memory performance (proportion correct) on the probe trials in the identity only condition was significantly higher than chance (.50) at all lags, t(9) / /6.24, p B/.001 for lag 2 and p/.001 for lags 1, 3, and 4 (see Figure 3). For the identity bound to location condition, only lags 1 and 4 were significantly higher than chance (both p-values B/.01 and lowest t-value for lag 4), t(9) / /3.37, p /.008. Lags 2 and 3 were not significantly different from chance (p s/.09). Memory performance was entered into an ANOVA with lag (1, 2, 3, 4) as a within-subject factor and condition (identity only, identity bound to location) as a between-subjects factor. Performance was higher in the identity only condition (M /70%) than the identity bound to location condition (M /59%), F (1, 18) /7.16, MS /0.24, p /.02. Lag had no effect 1 Proportion correct 0.9 0.8 0.7 0.6 0.5 0.4 0 1 2 3 4 Lag Identity only Identity bound to location Figure 3. Memory performance in Experiment 1. 5 IDENTITY MEMORY DURING SEARCH 11 on memory performance, F (3, 54) / 1.95, MS / 0.02, p / .13, and the interaction between lag and condition was nonsignificant, F (3, 54) / .19, MS / 0.001, p / .91. Discussion Replicating previous research, attentional guidance by memory for the locations occurred for at least the last three items examined, demonstrated by low refixation rates for old items during visual search. When asked to explicitly identify these items, participants were 70% accurate when distinguishing an item they had looked at just one fixation prior to the memory test from an item that was never viewed during the trial, and about 60% accurate at distinguishing the last examined item from other previously examined items. Better memory performance when location information was not needed to accurately choose the identity of the probed item (identity only condition) suggests that identity and location information about a previously examined item are not necessarily linked in memory. Furthermore, in both conditions, memory performance did not change across lag, suggesting that more recently inspected items are remembered equally well as items examined several fixations prior to the memory test. EXPERIMENT 2a Although explicit memory performance in Experiment 1 was above chance, it is possible that the nature of the gaze contingent search paradigm used in Experiment 1 may have impaired memory performance. The removal of items from the screen during saccades may disrupt memory representations associated with the items (Henderson, 1994; Henderson & Anes, 1994; Henderson & Siefert, 2001; Kahneman et al., 1992). In Experiments 2a and 2b we examine explicit memory for the identities of previously examined items during search using a more traditional search procedure. In this search task all items remain on screen until a response is made. If memory was poor in Experiment 1 because removing items from the screen disrupted explicit access to their representations, then we would expect to find improved memory performance when all items remain on screen. In Experiment 2a the foil was always another previously examined item in the display and in Experiment 2b the foil was an item that was not presented in the display. Method Participants. Eleven students (8 females and 3 males) from George Mason University participated in this study for course credit. The average 12 BECK ET AL. age of the participants was 26 years. All participants demonstrated normal or corrected-to-normal visual acuity. Apparatus. A Power Macintosh G4 (Dual 1 GHz), equipped with a 20inch (viewable) ViewSonic P225fb with a resolution of 640 /480 pixels and capable of running at 120 Hz, running custom software was used to present the stimuli, control the timing of experimental events, and record participants’ response times. This computer was networked to a Dell Pentium 4 machine that collects eye-tracking data in conjunction with an Eyelink II system (same system as used in Experiment 1). The chinrest was located 61 cm from the monitor which subtended approximately a 30.28 visual angle. Stimuli and procedure. Participants viewed static search displays containing small white letters (0.668 /0.568) on a black background. Participants were asked to search for an S or a U target among distractor letters comprising of the remaining letters of the alphabet from A to V. Participants were asked to press one of two buttons depending on the identity of the target. The trial started with participants fixating a white fixation point at the centre of the screen and pressing the spacebar. The fixation point was then removed and a search array of 12 items appeared on the screen (11 distractors and 1 target). When the items were not fixated, white circular placeholders (Gaussian function, SD/0.38) covered each item. When the direction and speed of a saccade indicated that a specific item was about to be fixated the placeholder was removed to reveal the item. Therefore, participants had to fixate each item to determine its identity. On most trials (67%), the trial continued until the target was found. On the remaining trials (33%), the trial was stopped after the fifth fixation on an item; a red circle (diameter /2.48) appeared around one of the locations. All the items remained on screen covered by placeholders, regardless of fixation position. One letter appeared at the top left of the circle and one letter appeared at the top right of the circle. One letter was always the letter that appeared at that location, and this letter could occur equally often to the left or right of the probe circle. The other letter was a foil letter, a letter that participants had fixated during the trial, but occurred at a location other than the probe location. The foil was randomly chosen from the list of items fixated previously. In each condition, the probed letter could be the letter fixated one, two, three, or four fixations back (lag). Participants were asked to choose which item was at the probed location by pressing one of two buttons on the keyboard. Participants were instructed that the search task was the primary task and that the majority of trials would be search trials. Participants completed two blocks of 160 trials separated by a self-timed break. IDENTITY MEMORY DURING SEARCH 13 Results and discussion Search performance. Participants with accuracy lower than 90% were excluded from analysis (n /1). Overall search accuracy for the remaining 10 subjects was 97%. Eye movements. In order to examine memory for the location of previously examined letters, we compared the rate of refixations on previously examined items in the nonprobe trials to the rates that would be expected in a memoryless search. Similarly to Peterson et al. (2001), a Monte Carlo simulation of memoryless search was conducted (i.e., the probability of randomly revisiting an item on each fixation was calculated based on the number of items available for revisiting). Eleven ‘‘subjects’’ were run through a simulation of 143 trials (the average number of nonprobe trials) at a set size of 12 with the following constraints. Items were randomly selected for each fixation excluding the most recently fixated item. If a selected item had been previously selected during the trial (a revisit), the number of items between this selection and the previous was recorded (revisit lag). The probability of revisiting an item was calculated by dividing the number of revisits at each revisit lag by the number of gazes in the trial. The probability of revisiting items in the observed data of the nonprobe trials was similarly calculated by dividing the number of revisits at each revisit lag by the number of gazes in a trial. Observed data was compared to predicted data (memoryless model) using multiple t-tests. The observed probability of refixations was significantly lower than that of the memoryless model for revisit lags 1 /4, smallest t(11) /6.58 at revisit lag 1, all ps B/.001 (see Figure 4). These data suggest that there is memory for the locations of at least the last four examined items. Explicit memory performance. One-sample t-tests revealed that memory performance was significantly higher than chance (50%) at lags 1, 2, and 3, t(9) /5.785, p B/ .01, t(9) /3.55, p B/.01, and t (9) /2.79, p/.02, for lags 1, 2, and 3, respectively, but memory performance at lag 4 was not significantly different from chance, t (9) /1.49, p /.17. Memory performance on the probe trials (percentage correct) were entered into an ANOVA with lag (1, 2, 3, 4) as a within-subjects factor. Memory performance varied across lag, F (3, 27) /5.34, MS /0.05, p /.005. A trend analysis used to further examine the main effect of lag on memory performance, revealed a significant linear trend, F (1, 9) /14.79, MS /0.12, p/.004, and accounted for 62% of the within-subjects variation for memory performance. As can be seen in Figure 5 this trend was characterized by a decrease in memory performance as lag increased. 14 BECK ET AL. 0.08 Memoryless Observed Proportion of revists 0.07 0.06 0.05 0.04 0.03 0.02 0.01 0 0 1 2 3 4 5 Lag Figure 4. Proportion of revisits to old items in Experiment 2a (observed) and the proportion predicted by the memoryless search model (memoryless). As in Experiment 1, participants used memory of the locations of previously examined items to bias fixations toward new items, and explicit memory performance for the identities of the items was above chance for the three most recently examined items. Furthermore, the linear trend for memory performance across lag suggested that memory for an examined 1 Proportion correct 0.9 0.8 0.7 0.6 0.5 0.4 0 1 2 3 Lag Figure 5. Memory performance in Experiment 2a. 4 5 IDENTITY MEMORY DURING SEARCH 15 item’s identity rapidly fades as more items are examined. In Experiment 2b we examine memory for previously examined items not bound to location. The foil is always an item that did not appear during the trial. Improved memory performance would suggest that memory for the items is available but it is not bound to the locations of the items. EXPERIMENT 2b Method Participants. Twenty-four students (23 females and 1 male) from George Mason University participated in this study for course credit. The average age of the participants was 20 years. All participants demonstrated normal or corrected-to-normal visual acuity. Stimulus and procedures. All stimuli and procedures were the same as in Experiment 2a except on memory probe trials; the foil was a letter that had not been viewed during the trial. Results and discussion Search performance. Participants with accuracy lower than 90% were excluded from analysis (n /1). Overall search accuracy for the remaining 23 participants was 97%. Eye movements. A Monte Carlo simulation of memoryless search was conducted for 24 ‘‘subjects’’, 143 trials (the average number of nonprobe trails), and a set size of 12 with the same constraints reported in Experiment 2a. Observed data was compared to predicted data (memoryless model) using multiple t-tests. The observed probability of refixations was significantly lower than that of the memoryless model for revisit lags 1/4, smallest t(44) / /15.17 at revisit lag 1, all ps B/.001 (see Figure 6). These data suggest that there is memory for the locations of at least the last four examined items. Explicit memory performance. One-sample t-tests revealed that memory performance was significantly higher than chance (50%) at all lags, smallest t(22) /9.5 at lag 4, p B/.001. Memory performance on the probe trials (percentage correct) were entered into an ANOVA with lag (1, 2, 3, 4) as a within-subject factor. Memory performance varied across lag, F (3, 66) / 11.563, MS /0.08, p B/.001. A trend analysis used to further examine the main effect of lag on memory performance, revealed a significant linear trend, F (1, 22) /19.88, MS/0.19, p B/.001, and accounted for 48% of the 16 BECK ET AL. 0.08 Memoryless Observed Proportion of revisits 0.07 0.06 0.05 0.04 0.03 0.02 0.01 0 0 1 2 3 4 5 Lag Figure 6. Proportion of revisits to old items in Experiment 2b (observed) and the proportion predicted by the memoryless search model (memoryless). within-subjects variation for memory performance. There was also a significant quadratic trend, F(1, 22) / 7.67, MS / 0.03, p / .01, accounted for 26% of the within-subjects variation for memory performance. As can be seen in Figure 7, these trends are characterized by an initial strong decrease in memory performance as lag increased that then flattens out for the longer lags. Fixations were again biased away from previously examined items demonstrating the memory for the locations of previously examined items. Explicit memory performance for previously examined item’s identities was significantly above chance for all lags. As in Experiment 1, when memory is tested for identities not bound to location, performance is better than when it is bound to location (Experiment 2a). In both Experiment 2a and 2b, there was a linear trend for memory performance across lag. Memory performance was best for the items most recently examined, suggesting that memory for the identity of previously examined items fades as more items are examined. EXPERIMENT 2c Memory performance using a more traditional search task improved memory performance on average from 59% (identity bound to location condition of Experiment 1) to 63% (Experiment 2a) when location memory was required. When location memory was not required memory perfor- IDENTITY MEMORY DURING SEARCH 17 1 Proportion correct 0.9 0.8 0.7 0.6 0.5 0.4 0 1 2 3 4 5 Lag Figure 7. Memory data for Experiment 2b. mance improved from 70% (identity only condition from Experiment 1) to 77% (Experiment 2b) on average. In addition, there was a lag effect for previously examined items in the traditional search paradigm that was not present in Experiment 1. The lag effect appears to be driven mainly by better performance for the most recently examined item. This may have occurred because, in the traditional search paradigm (Experiments 2a and 2b), contextual information is available during the memory test. In Experiment 1, all items were removed from the screen during test and in Experiments 2a and 2b, each item was covered by a circular blob so that identity information was not available, but spatial contextual information was available. Therefore, spatial context may be helpful in accessing identity memories for recently examined items, thereby causing the lag effect in Experiments 2a and 2b. Experiment 2c tests this possibility by removing the spatial contextual cues during the memory test. Method Participants. Twenty-five students (22 females and 3 males) from George Mason University participated in this study for course credit. The average age of the participants was 20 years. All participants demonstrated normal or corrected-to-normal visual acuity. Stimulus and procedures. All stimuli and procedures were the same as in Experiment 2b except on memory probe trials all items were removed from 18 BECK ET AL. the screen for the memory test. The location of the memory probe item was circled and the probe and foil were presented on an otherwise blank screen. Results Search performance. All subjects had search accuracy above 90% and were included in the analysis. Overall search accuracy was 97%. Eye movements. Two subjects were excluded for missing eye movement data due to experimenter error. A Monte Carlo simulation of memoryless search was conducted for 25 ‘‘subjects’’, 143 trials, and a set size of 12 with the same constraints reported in Experiment 2a. Observed data was compared to predicted data (memoryless model) using multiple t-tests. The observed probability of refixations was significantly lower than that of the memoryless model for revisit lags 1/4, smallest t(38) / /9.18 at revisit lag 1, all ps B/.001 (see Figure 8). These data suggest that there is memory for the locations of at least the last four examined items. Explicit memory performance. One-sample t -tests revealed that memory performance was significantly higher than chance (50%) at all lags, smallest t (24) /6.43, p B/.001. Memory performance on the probe trials (percentage correct) were entered into an ANOVA with lag (1, 2, 3, 4) as a within-subject factor. Memory performance varied across lag, F (3, 73) /5.28, MS/0.04, p /.002. A trend analysis used to further examine the main effect of lag on 0.08 Memoryless Observed Proportion of revisits 0.07 0.06 0.05 0.04 0.03 0.02 0.01 0 0 1 2 3 4 5 Lag Figure 8. Proportion of revisits to old items in Experiment 2c (observed) and the proportion predicted by the memoryless search model (memoryless). IDENTITY MEMORY DURING SEARCH 19 1 Proportion correct 0.9 0.8 0.7 0.6 0.5 0.4 0 1 2 3 4 5 Lag Figure 9. Memory performance for Experiment 2c. memory performance, revealed a significant linear trend, F (1, 24) /15.72, MS /0.10, p /.001, and accounted for 39.6% of the within-subjects variation for memory performance. As can be seen in Figure 9 these trends are characterized by a decrease in memory from lags 1 and 2 to lags 3 and 4. Discussion Removing the context during the memory test did not eliminate the linear effect of lag. However, this linear effect in Experiment 2c was not driven by a drop in performance from the most recently examined item versus the other items. Rather the trend was driven by a drop in performance from lags 1 and 2 relative to lags 3 and 4. Furthermore, memory performance at lag 1 in Experiment 2c (M /77%) was significantly lower than performance at lag 1 in Experiment 2b (M /85%), t(46) /2.38, p/.02, but performance did not differ between experiments at any of the other lags, largest t(46) /1.27, p / .21 at lag 3. Therefore, it appears that contextual information available at test was at least partially responsible for better memory performance for the most recently examined item in Experiment 2b. GENERAL DISCUSSION The probability of maintaining the identities of the distractors during visual search in memory is above chance, and the identities of distractors do not 20 BECK ET AL. appear to be necessarily bound to memory for the distractors’ locations. Across different types of visual search tasks, explicit memory performance for the identities of previously examined distractors was on average around 66%. When memory was tested for only the identity of the previously examined distractors, performance was reliably above chance. However, when the memory test required knowledge of not only the distractors’ identities but also their locations, memory performance tended to be above chance for only the most recently examined item. Furthermore, providing contextual information during the testing phase improved memory performance for the most recently examined item. Memory for the identities of previously examined items was best when the most recently examined item was tested, contextual information was provided, and location memory was not required (Experiment 2b; M /85%). It is surprising that memory performance for the distractors’ identities is not higher given that the memory tests occurred during the search trials and that the participants were aware that the memory tests would occur. The memory performance levels in the current experiments for the identities of previously examined items are similar to those found by Williams et al. (2005). Participants in our experiments were aware that a memory test could occur while they were searching the stimuli, but participants in Williams et al.’s experiments were unaware until after completing all of the visual search trials that a memory test would be administered. Therefore, knowing that memory may be tested does not appear to improve memory performance, and the amount of memory available during the search task appears to be similar to the amount available in LTM. However, these possibilities need to be tested directly, because several other differences between our experiments and those used by Williams et al. may be responsible for the unexpected similarities in performance.2 Does visual search interfere with identity memory? Although explicit memory performance for distractor identities was generally above chance, it was still much lower than would be expected given that the items were examined within a few seconds before the test and 2 Williams et al. (2005) used a different type of search task than was used in the experiments reported here. Instead of asking participants to search for the presence of a target, they were asked to report how many targets were present (0, 1, 2, or 3). The search target was different than that given in the experiments reported here; participants were told to search for a specific object of a specific category (e.g., ‘‘yellow phone’’). Furthermore, distractors could be from the same specific category as the target (e.g., green phones). Any of these differences could have lead to better performance on the LTM test than would have been found using a search task like what was used in the experiments reported here. IDENTITY MEMORY DURING SEARCH 21 that the number of items tested was within the capacities of working memory (4 /5 objects: Cowan, 2001; Irwin, 1992; Irwin & Andrews, 1996; Luck & Vogel, 1997; Pashler, 1988; Phillips, 1974). When participants are given the sole task of memorizing the identity and locations of an array of objects, they are able to remember about ‘‘5 objects worth of information’’ (Irwin & Zelinsky, 2002). We used the same method of calculating the number of objects that can be remembered (capacity) for each of the experiments reported here.3 In Experiment 1, 3.0 objects’ worth of information was remembered when identity bound to location memory was needed and 3.8 objects’ worth when only identity memory was needed. When a more traditional search task was used, participants remembered 3.6 objects’ worth of information when only identity memory was needed (Experiment 2a) and 2.7 objects’ worth of information when identity bound to location memory was need (Experiment 2b). Similar capacity levels were found in Experiment 2c when contextual information was not provided at test and only identity memory was needed (3.3 objects’ worth of information). Across the experiments reported here roughly 3 objects’ worth of information was retained during the visual search tasks. Thus, although the memory task used in this experiment was also an intentional task, presenting it secondary to the visual search task resulted in the loss of 2 objects’ worth of identity memory. The intentional memory task used here may have inflated this estimate. It is possible that during typical visual search tasks (when identity memory is not explicitly probed), the amount of explicit identity memory is even lower than we found here. Visual search tasks could lower the amount of distractor identity information retained in memory because search tasks generally encourage memory for the locations of the nontarget items rather than their identities. First, the efficient strategy of biasing fixations away from old items may contribute to reduced memory for distractor identities. Second, visual search tasks may not require remembering the identities of the distractors because only task relevant identities are remembered. Finally, visual search tasks may result in memory representations of distractor identities that are not always retrievable through explicit memory tests. Each of these possibilities is discussed below. 3 The number of objects’ worth of information stored in memory during visual search was calculated similarly to how it was calculated in Irwin and Zelinsky (2002). We corrected for guessing (Busey & Loftus, 1994) by applying the formula p / (x / g)/(1 / g), where x is the percentage correct averaged across all lags, g is the probability of guessing (g / 1/n, were n equals the number of objects examined prior to the memory test, six in Experiment 1 and five in Experiments 2a, b, and c), and p is the percentage correct after correcting for guessing. P was then multiplied by the total number of objects examined (n) for an estimate of the number of objects remembered during visual search (Sperling, 1960). 22 BECK ET AL. Infrequent reinspections of old items during search may prevent the build up of strong memories for distractor identities. In all of the experiments presented here, fixations were biased away from old distractors. Therefore, the processing time allotted to each distractor is generally limited to the duration of one fixation (less than 300 ms). Distractors that are fixated more frequently and receive longer total fixation durations are more likely to be remembered (Williams et al., 2005). Specifically, as the number of fixations on a distractor increased, LTM memory performance increased as well. Therefore, the memory for locations used to prevent distractors from being reexamined during visual search may be hindering the ability to build up explicit identity memory for the distractors by limiting the number of times distractors are examined. Explicit memory for the identity of the distractors may be lower than found in nondual task intentional memory tests because the distractors are not directly relevant to the task of responding to the target’s identity or presence. Targets are special because they are more important to the task and are therefore likely to be processed more deeply and for longer periods of time (Oulasvirta, 2004). In addition, participants are able to search for the presence of more than one target with high accuracy suggesting they are able to remember spatial and temporal information about multiple targets (Gibson, Li, Skow, Brown, & Cooke, 2000; Thornton & Horowitz, 2004). Williams et al. (2005) found LTM performance of about 85% for targets versus about 50/60% for distractors. This suggests that memory for distractors in traditional search may be lower than memory for distractors because only information that is directly related to the task is remembered. The combination of lower memory for identity than found in nondual task intentional memory tests and results from previous research demonstrating that changing the identities of distractors during search does not disrupt search performance (Beck et al., in press) suggests that the identities of the distractors are not always encoded during visual search. Alternatively, it is possible that memory for the distractors’ identities is encoded, but that the memory is implicit and not used to guide attention during the visual search task. This predicts that our explicit measure of memory would be unable to retrieve the memory, and because the implicit memory is not used to guide attention, changing the identity would not disrupt the search process. Even when there is no declarative memory for an item, the item may be processed such that when the item is encountered later, processing of the item is facilitated (priming; Levy, Stark, & Squire, 2004). However, even implicit memory for distractor identity may be low during visual search because implicit memory can also be affected by task relevance (Hayhoe, 2000; Hayhoe, Bensinger, & Ballard, 1998; Holbrook, Bost, & Cave, 2003; Land, Mennie, & Rusted, 1998; Triesch, Ballard, Hayhoe, & Sullivan, 2003). IDENTITY MEMORY DURING SEARCH 23 Memory for identity versus location The lack of revisits to previously examined items reported here supports earlier findings suggesting that memory in visual search is based on the locations of already examined items (Chun & Jiang, 1997; Oh & Kim, 2004; Peterson et al., 2001; Woodman & Luck, 2000). In comparison to the estimates reported here of roughly three items’ worth of identity information remembered during visual search, estimates of the number of items for which location information is remembered is about five items (after factoring out contributions of scan-path strategies; McCarley et al., 2003; Peterson, Beck, & Vomela, 2004). However, the research reported here suggests that this information is not necessarily retrieved together. Visual search tasks may use the attentional resources necessary for binding for other processes immediately relevant to the search task, and therefore the memories are stored separately and not bound together (Wheeler & Treisman, 2002). Alternatively, the retrieval of location information about an item may inhibit the retrieval of identity information (Anderson, Bjork, & Bjork, 1994). In conclusion, memory for the identities of distractors during visual search is above chance but lower than memory during nondual task intentional memory tests (three objects’ worth of information versus five objects’ worth respectively). This indicates that some identity information is retained, but the amount retained is less than would be expected by the limits of working memory. This may occur because the location of distractors is relevant to the visual search task but their identities are not. Furthermore, the retrieval of location information is not necessarily linked to the retrieval of identity information during visual search. REFERENCES Anderson, M. C., Bjork, R. A., & Bjork, E. L. (1994). Remembering can cause forgetting: Retrieval dynamics in long-term memory. Journal of Experimental Psychology: Learning, Memory, and Cognition , 20 (5), 1063 /1087. Beck, M. R., Peterson, M. S., & Vomela, M. (in press). Where but not what is remembered during visual search. Journal of Experimental Psychology: Human Perception and Performance. Boot, W. R., McCarley, J. S., Kramer, A. F., & Peterson, M. S. (2004). Automatic and intentional memory processes in visual search. Psychonomic Bulletin and Review, 11 (5), 854 / 861. Busey, T., & Loftus, G. (1994). Sensory and cognitive components of visual information acquisition. Psychological Review, 101 , 446 /469. Castelhano, M. S., & Henderson, J. M. (2005). Incidental visual memory for objects in scenes. Visual Cognition , 12 , 1017 /1040. Chun, M. M., & Jiang, Y. (1998). Contextual cueing: Implicit learning and memory of visual context guides spatial attention. Cognitive Psychology, 36 , 28 /71. 24 BECK ET AL. Cowan, N. (2001). The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and Brain Sciences, 24 , 87 /185. Dodd, M. D., Castel, A. D., & Pratt, J. (2003). Inhibition of return with rapid serial shifts of attention: Implications for memory and visual search. Perception and Psychophysics, 65 (7), 1126 /1135. Gibson, B., Li, L., Skow, E., Brown, K., & Cooke, L. (2000). Searching for one versus two identical targets: When visual search has a memory. Psychological Science, 11 (4), 324 /327. Hayhoe, M. M. (2000). Vision using routines: A functional account of vision. Visual Cognition , 7 , 43 /64. Hayhoe, M. M., Bensinger, D. G., & Ballard, D. H. (1998). Task constraints in visual working memory. Vision Research , 38 (1), 125 /137. Henderson, J. M. (1994). Two representational systems in dynamic visual identification. Journal of Experimental Psychology: General , 123 (4), 410 /426. Henderson, J. M., & Anes, M. D. (1994). Roles of object-file review and type priming in visual identification within and across eye fixations. Journal of Experimental Psychology: Human Perception and Performance, 20 (4), 826 /839. Henderson, J. M., & Siefert, A. B. C. (2001). Types and tokens in transsaccadic object identification: Effects of spatial position and left /right orientation. Psychonomic Bulletin and Review, 8 (4), 753 /760. Holbrook, J. B., Bost, P. R., & Cave, C. B. (2003). The effects of study-test relevance on perceptual repetition priming. Memory and Cognition , 31 (3), 380 /392. Hollingworth, A., & Henderson, J. M. (2002). Accurate visual memory for previously attended objects in natural scenes. Journal of Experimental Psychology: Human Perception and Performance, 28 , 113 /136. Horowitz, T. S., & Wolfe, J. M. (1998). Visual search has no memory. Nature, 394 , 575 /577. Irwin, D. E. (1992). Memory for position and identity across eye movements. Journal of Experimental Psychology: Learning, Memory, and Cognition , 18 (2), 307 /317. Irwin, D. E., & Andrews, R. V. (1996). Integration and accumulation of information across saccadic eye movements. In T. Inui & J. L. McCleland (Eds.), Attention and performance XVI: Information integration in perception and communication (pp. 125 /155). Cambridge, MA: MIT Press. Irwin, D. E., & Zelinsky, G. J. (2002). Eye movements and scene perception: Memory for things observed. Perception and Psychophysics, 64 (6), 882. Kahneman, D., Treisman, A., & Gibbs, B. J. (1992). The reviewing of object files: Object-specific integration of information. Cognitive Psychology, 24 , 175 /129. Klein, R. M., & MacInnes, W. J. (1999). Inhibition of return is a foraging facilitator in visual search. Psychological Science, 10 (4), 346 /352. Kristjansson, A. (2000). In search of remembrance: Evidence for memory in visual search. Psychological Science, 11 (4), 328 /332. Land, M., Mennie, N., & Rusted, J. (1998). Eye movements and the role of vision in activities of daily living: Making a cup of tea. Investigative Ophthalmology and Visual Science, 39 , S457. Levy, D. A., Stark, C. E. L., & Squire, L. R. (2004). Intact conceptual priming in the absence of declarative memory. Psychological Science, 15 (10), 680 /686. Luck, S. J., & Vogel, E. K. (1997). The capacity of visual working memory for features and conjunctions. Nature, 390 , 279 /281. McCarley, J. S., Wang, R. F., Kramer, A. F., Irwin, D. E., & Peterson, M. S. (2003). How much memory does oculomotor search have? Psychological Science, 14 (5), 422 /426. Müller, H. J., & von Muhlenen, A. (2000). Probing distractor inhibition in visual search: Inhibition of return. Journal of Experimental Psychology: Human Perception and Performance, 26 (5), 1591 /1605. IDENTITY MEMORY DURING SEARCH 25 Oh, S. H., & Kim, M. S. (2004). The role of spatial working memory in visual search efficiency. Psychonomic Bulletin and Review, 11 , 275 /281. Oulasvirta, A. (2004). Task demands and memory in web interaction: A levels of processing approach. Interacting with Computers, 16 , 217 /241. Pashler, H. (1988). Familiarity and visual change detection. Perception and Psychophysics, 44 , 369 /378. Peterson, M. S., Beck, M. R., & Vomela, M. (2004, May). The guidance of attention by retrospective and prospective memory during visual search . Paper presented at the conference of the Vision Sciences Society, Sarasota, FL. Peterson, M. S., Kramer, A. F., Wang, F. R., Irwin, D. E., & McCarley, J. S. (2001). Visual search has memory. Psychological Science, 12 (4), 287 /292. Phillips, W. A. (1974). On the distinction between sensory storage and short-term visual memory. Perception and Psychophysics, 16 , 283 /299. Sperling, G. (1960). The information available in brief visual presentations. Psychological Monographs, 74 (11), 1 /29. Standing, L. (1973). Learning 10,000 pictures. Journal of Experimental Psychology, 25 , 207 / 222. Takeda, Y. (2004). Search for multiple targets: Evidence for memory-based control of attention. Psychonomic Bulletin and Review, 11 (1), 71 /76. Takeda, Y., & Yagi, A. (2000). Inhibitory tagging in visual search can be found if search stimuli remain visible. Perception and Psychophysics, 62 (5), 927 /934. Thorton, I. M., & Horowitz, T. S. (2004). The multi-item localization (MILO) task: Measuring the spatio-temporal context of vision for action. Perception and Psychophysics, 66 (1), 38 /50. Triesch, J., Ballard, D. H., Hayhoe, M. M., & Sullivan, B. T. (2003). What you see is what you need. Journal of Vision , 3 , 86 /94. Wheeler, M. E., & Treisman, A. (2002). Binding in short-term visual memory. Journal of Experimental Psychology: General , 131 (1), 48 /64. Williams, C. C., Henderson, J. M., & Zacks, R. T. (2005). Incidental visual memory for targets and distractors in visual search. Perception and Psychophysics, 67 (5), 816 /827. Wolfe, J. M. (2003). Moving towards solutions to some enduring controversies in visual search. Trends in Cognitive Sciences, 7 (2), 70 /76. Wolfe, J. M., Oliva, A., Butcher, S. J., & Arsenio, H. C. (2002). An unbinding problem? The disintegration of visible, previously attended objects does not attract attention. Journal of Vision , 2(3), 256 /271. Retrieved 8/9/04, from http://journalofvision.org/2/3/5. Woodman, G. F., & Luck, S. J. (2004). Visual search is slowed when visuospatial working memory is occupied. Psychonomic Bulletin and Review, 11 , 269 /274. Manuscript received June 2005 Manuscript accepted January 2006 PrEview proof published online month/year
© Copyright 2026 Paperzz