This article appeared in a journal published by Elsevier. The attached copy is furnished to the author for internal non-commercial research and education use, including for instruction at the authors institution and sharing with colleagues. Other uses, including reproduction and distribution, or selling or licensing copies, or posting to personal, institutional or third party websites are prohibited. In most cases authors are permitted to post their version of the article (e.g. in Word or Tex form) to their personal website or institutional repository. Authors requiring further information regarding Elsevier’s archiving and manuscript policies are encouraged to visit: http://www.elsevier.com/authorsrights Author's personal copy Developmental Cognitive Neuroscience 6 (2013) 102–112 Contents lists available at ScienceDirect Developmental Cognitive Neuroscience journal homepage: http://www.elsevier.com/locate/dcn Relational congruence facilitates neural mapping of spatial and temporal magnitudes in preverbal infants Daniel C. Hyde a,∗ , Chris L. Porter b , Ross Flom c , Sarah A. Stone b a b c Department of Psychology, University of Illinois at Urbana-Champaign, United States School of Family Life, Brigham Young University, United States Department of Psychology, Brigham Young University, United States a r t i c l e i n f o Article history: Received 11 March 2013 Received in revised form 24 June 2013 Accepted 12 August 2013 Keywords: Event-related potentials Infancy Perception Intersensory Multisensory Sensory integration Attention Memory Auditory processing Visual processing PSW Nc a b s t r a c t Mental representations of space, time, and number are fundamental to our understanding of the world around us. It should come as no surprise that representations of each are functional early in human development, appear to share a common format, and may be maintained by overlapping cortical structures. The consequences of these similarities for early learning and behavior are poorly understood. We investigated this issue by assessing neurophysiological processing of audio-visual temporal and spatial magnitude pairs using event-related potentials (ERPs) with young infants. We observed differential early processing and later enhanced attentional processing for pairings of spatial and temporal magnitudes that were relationally congruent (short visual character paired with a short auditory tone or long visual character paired with a long auditory tone) compared to the same stimuli paired in a relationally incongruent manner (short visual character with the long auditory tone or long visual character with a short tone). Unlike previous studies, these results were not dependent on a redundancy of information between the senses or an alignment of congruent magnitude properties within a single sense modality. Rather, these results demonstrate that mental representations of space and time interact to bias learning before formal instruction or the acquisition of spatial language. © 2013 Elsevier Ltd. All rights reserved. 1. Introduction Immanuel Kant (1781/1998) proposed that a primitive understanding of space, time, and number is innate and forms the basis of later learning and experience. Kant’s philosophical assertion has garnered support in light of research showing that preverbal human infants, as well as many non-human animals, have the cognitive capacity and underlying neural structure to support mental representation of space, time, and number (Dehaene and Brannon, ∗ Corresponding author at: University of Illinois at Urbana-Champaign, 621 Psychology Building, 603 East Daniel Street, Champaign, IL 61821, United States. Tel.: +1 217 300 0382. E-mail addresses: [email protected], [email protected] (D.C. Hyde). 1878-9293/$ – see front matter © 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.dcn.2013.08.003 2010; Feigenson, 2007; Lourenco and Longo, 2011). Furthermore, research with human adults demonstrates that mental representations of space, time, and number interact with each other (e.g. Casasanto and Boroditsky, 2008; Cohen Kadosh et al., 2007; Pinel et al., 2004; Walsh, 2003). For instance, Casasanto and Boroditsky (2008) found that judgments made regarding the temporal duration of visual stimuli are influenced by the physical length of the stimuli. Likewise, past work (e.g. Algom et al., 1996; Cohen Kadosh et al., 2007; Tzelgov et al., 1992) with the “numerical Stroop task” has shown that adults’ judgments of numerical size (e.g. which is numerically larger?) are influenced by the physical size of the Arabic digits (i.e. 2 vs. 7). These interactions suggest that representations of one dimension (e.g. size) activate representations in the other (e.g. number). Unfortunately, the origin of these interactions remains hotly debated and largely unknown. One possibility is Author's personal copy D.C. Hyde et al. / Developmental Cognitive Neuroscience 6 (2013) 102–112 that interactions between distinct representations of space, time, and number arise as a result of repeated associations between particular pairings of space, time, and/or number in the context of sensory-motor experience, language, or culture (Boroditsky, 2000, 2011; Gelman and Williams, 1998; Johnson, 2003; Piaget, 1927/1969). Alternatively, interactions between representations of space, time, and number might arise because of an innate link, heretofore referred to as a functional overlap, between these dimensions (e.g. Cantlon et al., 2009; Pinker, 1997; Srinivasan and Carey, 2010; Walsh, 2003). Some have hypothesized that innate functional overlap could be realized as an initially privileged relationship between distinct representations of space, time, and number (e.g. van Marle and Wynn, 2006), while others hypothesize that innate functional overlap could be realized as an generalized magnitude system, which may later become differentiated over development (e.g. Lourenco and Longo, 2010; or see Lourenco and Longo, 2011 for a review). In either case, the current study tests for the presence of functional overlap in 5-month old infants, before associations between space, time, and number could likely be culturally, linguistically, or educationally constructed. Three lines of evidence suggest a similarity, or overlap, in the way space, time, and number are mentally represented. First, these representations appear to share a common analog format, where quantities are represented as approximate physical magnitudes proportional to that being represented (see Meck and Church, 1983; Walsh, 2003). Second, representations of space, time, and number have common or signature behavioral limits (see Dehaene and Brannon, 2010 for a review). For example, the ability of human infants, human adults, and many non-human animals to compare two magnitudes, whether they be spatial, temporal, or numerical, is limited by the ratio of the two magnitudes being compared rather than the absolute difference between them (e.g. Brannon et al., 2006, 2008; Lipton and Spelke, 2003; van Marle and Wynn, 2006). Infants of the same age make comparisons at about the same level of precision, or ratio, regardless of the magnitude domain (e.g. Brannon et al., 2006, 2007; Feigenson et al., 2004). While the particular precision of each domain diverges in later childhood, all domains show continued gains in precision into adulthood (e.g. Droit-Volet et al., 2008; Halberda and Feigenson, 2008). Third, representations of space, time, and number have also been hypothesized to share a common neural substrate (Fias et al., 2003; Izard et al., 2009; Meck and Church, 1983; Walsh, 2003). For example, neuroimaging and neurophysiological work suggests overlap in regions of the intraparietal sulcus (IPS) involved in the representation and/or comparisons of number and physical size (i.e. area or length) (e.g. Pinel et al., 2004; Cohen Kadosh et al., 2007; Cohen Kadosh and Henik, 2006). In sum, while evidence suggests behavioral and neural similarities in the way space, time, and number are represented, the functional significance of this similarity for learning remains unclear. In the current study, we ask if similarity between representations of space and time biases early learning. Specifically, can young 5- to 6-month-olds learn magnitude relationships between dimensions of space and time more 103 readily when they are relationally congruent compared to when they are relationally incongruent? Several recent studies suggest that the overlap between representations of space, time, and number plays a functional role in learning, especially early in development (deHevia and Spelke, 2010; Lourenco and Longo, 2010; Mix et al., 1999, 2002; Srinivasan and Carey, 2010). For example, work with pre-school and elementary school children suggests that early numerical calculation abilities may be aided by corresponding cues to spatial magnitude, or extent (Mix et al., 2002). Similarly, understanding of fractions may be facilitated by overlap in the way in which numerical and spatial magnitudes are represented (e.g. Mix et al., 1999). Again, like with human adults, explicit instruction, linguistic metaphor, or cultural experience may explain the facilitation of learning of children in these contexts. Recent work with pre-verbal infants suggests the learning may be facilitated between dimensions before such experience. For example, Lourenco and Longo (2010) demonstrated that infants are able to form interchangeable associations between number and space and time in learning which stimuli go together. Additional work shows that infants learn associations between relationally congruent line lengths and numerical quantities more readily (smaller numbers of objects associated with shorter line lengths/larger numbers of items associated with longer lines) than they learn relationally incongruent pairings of the same stimuli (shorter lines with larger numbers of items/longer lines with smaller number of items) (deHevia and Spelke, 2010). In both of these experiments (i.e. deHevia and Spelke, 2010; Lourenco and Longo, 2010), however, pairings were made only over visual dimensions. Thus, it is unclear whether infants made the associations because of functional overlap between abstract representations of space, time, and number, an abstract generalized magnitude, or whether the results were due to common visual-specific properties of the stimuli. In an attempt to discern whether the ability to learn associations between space and time was a product of functional overlap between abstract mental representations or whether previous results are due to common visual properties, Srinivasan and Carey (2010) required such associations between space and time to be made across two different sense modalities. Srinivasan and Carey (2010) found that infants are better at learning associations between visually presented length (space) and temporal duration of an auditory tone (time) when they are relationally congruent (e.g. relatively long duration tone with long line, short duration tone with short line) than when they are relationally incongruent. Moreover, Srinivasan and Carey (2010) argue that the functional overlap between abstract representations of space and time allow the correspondences between them to be spontaneously aligned, facilitating the learning for relationally congruent pairings but not for incongruent pairings. While the results of Srinivasan and Carey (2010) support their position, it should be noted that temporal magnitude information was conveyed both aurally and visually. That is, the duration of stimulus was redundant across the senses, or could be obtained both visually and aurally. Spatial and temporal information was also presented in Author's personal copy 104 D.C. Hyde et al. / Developmental Cognitive Neuroscience 6 (2013) 102–112 perfect temporal synchrony from stimulus onset to stimulus offset. This is significant because related work has shown that, within the domain of number as well as other domains, redundant and temporally synchronous information conveyed across multiple senses enhances learning compared to bimodal presentation without redundancy (e.g. Bahrick and Lickliter, 2000; Jordan et al., 2008; Flom and Bahrick, 2010). Therefore, it is unclear if the ability to learn congruent pairings in Srinivasan and Carey (2010) was dependent on the redundancy of temporal magnitude information across both sense modalities (see Bahrick and Lickliter, 2000; Bahrick et al., 2004 for reviews). Alternatively, despite using audio-visual stimuli, differential learning between congruent and incongruent pairs in Srinivasan and Carey (2010) could also have taken place purely within the visual domain. Visual length and visual duration changed as a function of condition. It is plausible that the duration of visual stimulation was mapped to the spatial extent of the visual stimulation and this mapping was easier when the temporal and spatial information was congruent compared to when it was incongruent. In this case, auditory information could be completely ignored and still produce the observed results. If this is so, it is still unclear whether differential learning occurs as a result of functional overlap in representations of space and time abstracted away from basic sensory properties or whether differential learning can be explained based on basic sensory correspondences, or lack thereof, between temporal and spatial properties of primary visual stimulation. The current study was designed to overcome the limitations of previous work by using audio-visual stimuli with asynchronous offsets, thereby presenting non-redundant audio and visual magnitude information for length and time in distinct sense modalities (see Fig. 1). In this way, the learning of space–time magnitude pairings could not be explained by temporal synchrony of information or redundancy of spatial and temporal information across the senses. In addition, differential learning in the current study could not be explained by temporal and spatial magnitude correspondences within a single sense modality because visual–temporal information was held constant across all test conditions. Finally, while behavioral measures used within previous studies (e.g. deHevia and Spelke, 2010; Lourenco and Longo, 2010; Srinivasan and Carey, 2010) have revealed interesting patterns of results, they provide limited, if any, insight into the level of processing at which differential processing or learning may occur. It is possible, for example, that enhanced learning of congruent spatial–temporal pairings is due to facilitated early processing, more attention toward, and/or better memory for congruent pairings compared to incongruent pairings. The current study allowed for an assessment at each of these levels by employing event-related potentials (ERPs) to measure the brain response. Neurophysiological work with infants employing ERPs has documented early and mid-latency posterior activity related to processing of high-level properties of complex stimuli such as the configuration or structure of faces and objects (e.g. N290 and P400), mid-latency anterior activity related to attentional orienting (Nc), and late anterior slow wave activity related to memory (e.g. PSW, NSW, Fig. 1. Audiovisual stimulus pairings used for training and test phases of the experiment. All visual stimuli were presented for 1000 ms and auditory stimuli were presented for 250 or 750 ms. The audiovisual onset was always synchronous and offset was always asynchronous. (A) Short caterpillar paired with a short duration tone. (B) Long caterpillar paired with a long duration tone. (C) Short caterpillar paired with a long duration tone. (D) Long caterpillar paired with a short duration tone. or LPC) (e.g. Halit et al., 2004; Scott and Nelson, 2006; Scott, 2011; Carver et al., 2003; de Haan and Nelson, 1997; Nelson, 1994; Richards, 2003; Grossmann et al., 2009; Quinn et al., 2006). Given the established functional properties of these distinct ERP components, measuring the infants’ neurophysiological response to relationally congruent and incongruent pairings may not only provide further evidence of functional overlap between representations of space and time, but may also help determine at what level differences arise. In order to investigate the neural basis of crossmodal magnitude learning, we recorded event-related potentials as 5-month old infants viewed asynchronous, non-redundant, audiovisual space–time pairings. Separate groups of infants were familiarized to audio-visual Author's personal copy D.C. Hyde et al. / Developmental Cognitive Neuroscience 6 (2013) 102–112 length-time pairings that were either relationally congruent (a long visual image was paired with a long auditory tone, or a short image was paired with a short tone) or relationally incongruent (short tone with long image or vice versa) (Fig. 1). The same auditory tones and visual cartoon characters were used in the training of both groups, but their relative pairings were different. This training phase was necessary to familiarize children with the range of magnitudes and particular pairings, as previous work has shown that young children may understand magnitudes better when defined in contrast to other magnitudes (rather than in isolation) (see Duffy et al., 2005). After a training phase, all participants were shown equal numbers of familiar and novel test pairings of the same visual–spatial lengths and auditory durations. Importantly, both groups of infants viewed the exact same test pairings of stimuli. Whether a given test pairing was familiar or novel was dependent on the training condition they had previously viewed (e.g. if trained with relationally congruent pairs, then relationally incongruent pairings were novel and vice versa). Thus, differences in brain responses to test trials between groups could not be readily explained by basic sensory differences between the test stimuli as all infants saw the same set of test stimuli. Instead, differences in neural processing between training conditions would likely to be due to the type of training they previously received (i.e. congruent or incongruent). Specifically, we examined differential learning in the training conditions by comparing event-related brain signatures of early high-level processing, mid-latency attentional orienting, and later memory updating. If infants are able to quickly and reliably learn both relationally congruent and incongruent pairings, then the brain processes reflecting early high level processing, attention, and/or memory should discriminate between familiar and novel stimulus pairings regardless of the training condition. However, if learning is facilitated by relationally congruent pairings compared to incongruent pairings, then brain processes reflecting high-level processing, attention, and/or memory should discriminate familiar and novel test pairings in those infants who were exposed to the congruent pairings during the training phase but not in those infants exposed to the incongruent pairings during the training phase. 2. Materials and methods 2.1. Participants Thirty-two five-month old infants (18 females; M age = 152 days) were randomly assigned to a relationally congruent (n = 16) or relationally incongruent training condition (n = 16). An additional 62 infants participated but were excluded from the analyses: 31 for early termination of experiment due to fussiness, crying, or inattentiveness and 31 for retaining too few good data segments after ERP artifact rejection. This attrition rate is within the range of attrition rates in other ERP studies of similar-aged infants (e.g. Hyde et al., 2010a,b; Hyde and Spelke, 2011; Izard et al., 2008; Quinn et al., 2006). 105 2.2. Stimuli and procedure Infants were initially exposed to 20 relationally congruent (10 spatially long image-temporally long tone/10 spatially short image-temporally short tone) or 20 relationally incongruent (10 spatially long image—temporally short tone/10 spatially short image—temporally long tone) audio-visual length-time pairings. Stimuli were presented in a pseudorandom order with the constraint that each (of the two) space–time pairings had to be presented before re-randomization occurred. After the training period, all infants were shown the same 44 test images of familiar (22 trials) or novel (22 trials) pairings of the same stimuli using the same pseudo-random procedure described above (each of the four pairings presented in a random order before re-randomization occurred). Length stimuli consisted of cartoon images of a yellow caterpillar of differing lengths (short and long) presented on a white background (Fig. 1). Stimuli were adapted with permission from Srinivasan and Carey (2010). Specifically, we adapted two of the more extreme versions of “long” and “short” visual and auditory stimuli that differed in magnitude by a 1:3 ratio, as previous infant studies have shown this difference to be sufficient for behavioral discrimination of spatial and temporal information at 5-months of age (Brannon et al., 2006, 2008). The short caterpillar consisted of three overlapping concentric body segments 2.5 in. in height and 2.5 in. in length. The long caterpillar consisted of 11 overlapping concentric body segments 2.5 in. in height but 7.75 in. in length. Both caterpillars had two eyes and antennas on the right-most segment to look like a head. Time stimuli consisted of the auditory tones lasting 250 ms (short temporal duration) and 750 ms (long temporal duration) played at approximately 70 decibels (Fig. 1). Congruent familiarization pairings consisted of equal numbers of trials with the longer caterpillar paired with the temporally longer tone and the shorter caterpillar paired with the temporally shorter tone. Incongruent familiarization pairings consisted of equal numbers of trials containing the shorter tone paired with the longer caterpillar and the longer tone paired with the shorter caterpillar. Two exemplars were used for each relationship (congruent or incongruent) to make it less likely that the average ERP response to familiar and novel pairings would be due to idiosyncratic sensory properties of a particular stimulus pairing used and more likely due to the overarching relational congruence of stimuli in relation to training. Critically, the temporal and spatial information were not redundant across the senses. Visual stimuli were always presented for 1000 ms and, while onset of the visual and auditory tones were synchronous, offset of the visual and auditory stimulus was never synchronous (determined by the length of the auditory stimulus: either 250 ms or 750 ms). All trials were followed by a blank-screen inter-stimulus interval that varied randomly in length from 900 to 1500 ms (1000 ms stimulation + 900–1500 ms ISI = 1900–2500 ms trial duration). Stimuli were presented from a 17-in. computer screen and computer speakers approximately 35–50 cm from the infant seated in a parent’s lap. The parent was instructed to look at the back of the infant’s head instead of the display in Author's personal copy 106 D.C. Hyde et al. / Developmental Cognitive Neuroscience 6 (2013) 102–112 order to blind them from the experimental condition that was being presented. Parents were also instructed not to speak to the infant in an attempt to reduce environmental interference. Looking behavior was monitored online, both training and test trials were only started when the infant was looking at the screen, and the experiment was paused when infants looked away for more than 1 s. If the baby became fussy, began to cry, or continually looked away, data collection was terminated. Halfway through the familiarization period and two times during the testing phase a short break was taken by showing the infant a picture of an animal with an accompanying animal sound (e.g. picture of a dog with a barking sound, picture of duck with quacking sound, etc.). This was done in an effort to reduce boredom and recapture infants’ attention to the video screen. This rest/break stimulus was presented at the same point in the experiment for all infants. 2.3. Data acquisition and reduction Infants’ heads were first measured and then, while the vertex was being located and marked on the baby’s scalp, the appropriate size sensor net was soaked in a potassium chloride solution. While the infant sat in a parent’s lap, the sensor net was systematically placed on the head with reference to the identified vertex and left and right mastoid bones. Before data collection, impedances were checked and maintained below 50 k. The ongoing EEG was then recorded from scalp locations using a 64 channel EGI HydroCel Sensor Net (Electrical Geodesics Inc., Eugene, OR) as infants were presented with stimuli in a dimly lit room. Data were recorded at 250 samples per second and digitally filtered online at 0.1–100 Hz, referenced to the vertex. Data from infants that completed the experiment were further processed in three steps. First, raw data were lowpass filtered at 30 Hz, segmented into epochs from 200 ms before to 1450 ms after stimulus onset, and baseline corrected to the 200 ms before the stimulus was presented. Second, two experienced staff members independently visually examined each epoch for artifacts. Any epoch containing an eye blink, eye movement, excessive noise, or more than 12 bad channels was rejected from further analysis. Questionable epochs were discussed between staff members until consensus could be reached. Any infant retaining less than 10 good epochs in either test type (familiar or novel test pairings) after artifact rejection was eliminated from the final analysis (see Section 2.1 for exact numbers). Remaining acceptable epochs were further processed by running an automated bad channel replacement algorithm based on spherical spline interpolation from surrounding good channels for trials containing less than 13 bad channels, then creating averages for each test type (familiar and novel test pairings) for each subject, re-referencing the data to the average reference, and again baseline correcting to 200 ms before stimulus onset to correct for any absolute amplitude differences created by processing. In addition to averages for each test type, a grand average for both test types (collapsed across familiar and novel pairings) was created for each training condition separately for visual inspection and peak latency analysis purposes. Individual participants included in the final analysis contributed no less than 10 trials per condition, and, on average, 12.328 trials per condition. There were no significant differences or interactions in the number of trials retained between Training Conditions or Test Types (all p’s > .18) (congruent training: M familiar test trials = 12.50, M novel test trials = 11.50; incongruent training: M familiar test trials = 12.75, novel test trials = 12.56). 2.4. Data analysis Early posterior processing was characterized as the average response over left, central and right posterior parietal electrode groups (EGI Hydrocel GSN electrodes: 31, 33-left; 36, 37-central; and 38, 40-right). Mid-to-late anterior processing was characterized as the average response over a fronto-central electrode group (sites 3, 6, 9). Electrode groups were chosen to characterize early posterior and mid-to-late anterior activity based on previous literature (see de Haan, 2007 or Reynolds and Richards, 2005 for reviews). We temporally focused our analysis from 250 ms post stimulus onset, given that none of the conditions could be distinguished before then (“short” auditory stimulus was 250 ms), to the end of the segment (1450 ms). Our analysis of the early posterior activity was restricted to rising second half of the first major posterior negativity (250–450 ms). Time windows of interest for the mid-to late latency anterior components (Nc, PSW) were chosen by using broad time windows to first acquire the peak latency of the grand mean (collapsed across familiar and unfamiliar test trial pairings) for each training condition (Nc = 450–950 ms; PSW = 950–1450 ms). Statistical analyses (see Section 3) revealed significant differences in peak Nc latency but not PSW latencies between training conditions (see Section 3). As a result, distinct, symmetrical time windows (−100/+100 ms) around the corresponding peak latencies for each training condition (congruent training: 599 ms; incongruent training: 711 ms) were used to characterize the Nc (congruent training: 499–699 ms; incongruent training: 611–811 ms) and a single, fixed, symmetrical time window (−100/+100) around the average peak latency (1131 ms) was used to characterize both training conditions for the PSW (average amplitude between 1031 and 1231 ms). Independent samples t-tests were used to test for latency differences between training conditions (collapsed across test type). ANOVAs with the between-subjects factor of Training Condition (relationally congruent vs. relationally incongruent) and the repeated within-subject factor of Test Type (familiar vs. novel pairing) statistically assessed mean component amplitude. Scalp Region was also a factor in the analysis of posterior processing (left, central, right electrode groups). Significant interactions were further examined post hoc using paired sample t-tests. 3. Results 3.1. Early posterior activity Test stimuli evoked a large negativity followed by a rising positivity over widespread posterior parietal scalp Author's personal copy D.C. Hyde et al. / Developmental Cognitive Neuroscience 6 (2013) 102–112 107 Fig. 2. Topographical scalp maps for components of interest as viewed from the top of the head (oriented with back of the head facing the bottom of the figure). Black dots represent electrode sites used to compute mean response. locations between about 175–700 ms after stimulus onset (Fig. 2). Timing and topography were consistent with the previously identified infant N290/P400 complex (e.g. de Haan, 2007). Waveform morphology, however, was slightly different (see Fig. 3) from that observed in previous studies using only visual stimulation (see de Haan, 2007 for review of infant visual-evoked posterior activity). Given that congruent and incongruent stimuli could only be differentiated after 250 ms and mid-latency anterior activity started around 450 ms, we were restricted to analyzing the rising (second) half of this early posterior negative component using the mean amplitude of a fixed time window (250–450 ms) averaged over left, midline, and right posterior sites. The amplitude analysis revealed a main effect of Scalp Region (F (2, 60) = 4.384, p < .05, 2p = .127), a main effect of Training Condition (F (1, 30) = 6.160, p < .05, 2p = .170), and an interaction between Training Condition and Test Type (F (1, 30) = 4.253, p < .05, 2p = .124) (all other p’s > .14). The left posterior parietal scalp group showed the most negative mean amplitudes during this time frame. Further post hoc analysis revealed that the interaction between Training Condition and Test Type on the second half of the early posterior negativity could be explained by a main effect of Test Type (F (1, 15) = 5.462, p < .05, 2p = .267) for those infants that were familiarized to the congruent pairing, with more negative amplitudes observed for familiar pairings compared to unfamiliar pairings, and no significant differences in Test Type for those infants who were familiarized to the incongruent pairings (all p’s > .35) (Fig. 3). 3.2. Mid-latency anterior negativity Test stimuli also evoked a mid-latency anterior negativity between 450 and 950 ms after stimulus onset (Fig. 2). Scalp topography and timing were consistent with the Author's personal copy 108 D.C. Hyde et al. / Developmental Cognitive Neuroscience 6 (2013) 102–112 Fig. 3. Mean posterior event-related potentials. (A) Average waveform over posterior channels between −200 and 1450 ms to familiar and novel test pairings for each training condition. The shaded region represents approximate time window of analysis. (B) Mean amplitude for the early posterior negativity to familiar and novel test pairings for each training condition. The “*” indicates a significant difference between conditions. previously identified Nc component (e.g. de Haan and Nelson, 1997; Nelson, 1994; Reynolds and Richards, 2005). Again, however, morphology was somewhat different (Fig. 4) than that observed in purely visual studies and more consistent with Nc morphology seen in studies with multisensory stimuli (Hyde et al., 2010b, 2011). An analysis of the peak latency of the Nc between 450 and 950 ms revealed a main effect of Training Condition (t (30) = −2.445, p < .05), with the Nc peaking significantly earlier (601 ms) for those infants presented with the congruent pairing during the training period compared to those infants presented with the incongruent pairing during the training phase (708 ms). Given the significant differences in Nc timing between groups, distinct time windows were used to characterize the mean amplitude of the Nc for each training group. Specifically, Nc was defined as a symmetrical 200-ms time window (−100 to +100 ms) surrounding the peak for each Training Condition over the fronto-central scalp group1 (congruent training 599–799 ms; incongruent training: 611–811 ms). An analysis of the mean Nc amplitude revealed a significant interaction between Training Condition and Test Type over anterior scalp (F (1, 30) = 4.183, p < .05, 2p = .122) (other p values > .43) (Fig. 4). Post hoc comparisons revealed that infants presented the congruent pairing during the training phase showed more negative frontal Nc responses to the familiar (congruent) pairings than to the novel (incongruent) pairings during the test phase (t (15) = −2.701, p < .05) (see Fig. 4). In contrast, infants 1 Given the sampling rate of 250 Hz per second or 1 sample every 4 ms, time windows of interest were centered (−100 to +100 ms) on the closest sampled time to the actual mean peak latency. presented incongruent pairings during the training phase showed no difference in mean Nc amplitudes to congruent and incongruent pairings at test (t (15) = .739, p > .47) (Fig. 4). 3.3. Late anterior positive component Some test stimuli evoked an anterior, late-going slow wave, which emerged once stimuli were no longer present (around 1000 ms) and returned to baseline around 1400 ms (see Fig. 2). Scalp timing, anterior topography, and wave morphology (see Figs. 2 and 4) were consistent with characterizations of the PSW or LPC in previous ERP studies with infants of this age (see de Haan, 2007 or Reynolds and Richards, 2005 for reviews). An analysis of peak latency over a broad time window (950–1450 ms), collapsed across test trial types, revealed no differences in timing between training conditions. As a result, a single symmetrical time frame (−100/+100 ms) around the average peak latency (1130 ms) was used to analyze the anterior positive slow wave for all conditions (1030–1230 ms). An analysis of the mean positive slow wave amplitude over anterior scalp revealed a significant interaction between Training Condition and Test Type (F (1, 30) = 5.631, p < .05, 2p = .158) (all other p’s > .82). Post hoc analysis revealed only a marginal difference between test types for those infants trained on the congruent test pairs (F (1, 15) = 3.393, p = .085, 2p = .184), with unfamiliar test trials eliciting a more positive amplitude than familiar test pairs, and no difference between test types for those infants trained on incongruent test pairs (F (1, 15) = 2.42, p > .14, 2p = .139). Given that the late slow wave activity is associated with memory updating for partially encoded stimuli and that Author's personal copy D.C. Hyde et al. / Developmental Cognitive Neuroscience 6 (2013) 102–112 109 Fig. 4. Mean anterior event-related potentials. (A) Average waveform over anterior channels between −200 and 1450 ms to familiar and novel test pairings for those subjects in the congruent training condition. The shaded region represents approximate time window of analyses. (B) Mean amplitude for the mid-latency Nc over anterior sites to familiar and novel test pairings for each training condition. The “*” indicates a significant difference between conditions. (C) Mean amplitude for the later positive slow wave (PSW) over anterior sites to familiar and novel test pairings for each training condition. The “*” indicates a significant response above baseline. (D) Average waveform over anterior channels between −200 and 1450 ms to familiar and novel test parings for those subjects in the incongruent training condition. The shaded region represents approximate time window of analyses. Author's personal copy 110 D.C. Hyde et al. / Developmental Cognitive Neuroscience 6 (2013) 102–112 a return to baseline is observed for full-encoded items and also for items that are not encoded at all (Nelson, 1994; Nelson and Collins, 1992; Nelson and de Regnier, 1992; Reynolds and Richards, 2005; Richards, 2003), we further analyzed, in a post hoc manner, the PSW relative to baseline using one-sample t-tests (against zero). This analysis revealed only the familiar test stimuli for those trained on incongruent pairings produced a PSW significantly greater than baseline (t (15) = 2.780, p < .05) (congruent training/familiar test, p = .79; congruent training/novel test: p = .063; incongruent training/novel test: p = .973). 4. Discussion We observed that the infant brain discriminates relationally congruent from relationally incongruent crossmodal space–time pairings when briefly familiarized with the relationally congruent pairings but not when familiarized with the relationally incongruent space–time pairings. These results are not due to low-level sensory differences between conditions, as infants in both training groups were presented with the same test stimuli. Furthermore, the design of our stimuli allowed us to rule out that differential processing was due to common spatio-temporal cues across a single sense modality, was dependent on redundant cues across multiple sense modalities, or was based on lower-level similarities between dimensions in the visual domain alone; instead, it appears that associations were made over amodal representations of space and time, abstracted away from non-redundant and asynchronous auditory and visual information. Therefore, we interpret the asymmetry in processing (i.e. congruent compared to incongruent pairings) between training groups to a functional overlap between mental representations of space and time, or a relational congruence bias, where structural similarities in the neural representation facilitate the automatic aligning and mapping between dimensions for relationally congruent pairings over relationally incongruent pairings, as suggested by Srinivasan and Carey (2010). By measuring the neurophysiological response, we observed that the asymmetry in processing begins, over posterior scalp sites, almost immediately after stimulus differences emerge. Neural processing was further differentiated both in timing and magnitude between training groups during mid-to-late latency time periods over anterior sites. Our functional interpretation is that, after a brief familiarization period to define relatively long and short magnitude pairings of auditory durations (tones) visual lengths (cartoon characters), relationally congruent pairings showed enhanced processing and attention orienting relative to incongruent pairings in those infants first familiarized to congruent pairings (long duration-long character/short duration-short character). In contrast, infants familiarized to incongruent pairings (short-long/long-short) showed no differences in early processing or attentional orienting between congruent and incongruent pairings in the test phase. Early posterior activity was temporally and topologically consistent with the infant N290/P400 complex typically observed for visual processing of faces and objects (e.g. Halit et al., 2004; Scott and Nelson, 2006; Scott, 2011). However, we are not committed to the idea that the observed early posterior differences have the same hypothesized ventral origin as those observed typically for visual processing of faces and objects (e.g. Halit et al., 2004; Scott and Nelson, 2006; Scott, 2011). Studies of numerical, spatial, and temporal processing, in contrast to face processing, typically activate dorsal cortical areas in the parietal cortex (Izard et al., 2008; Hyde et al., 2010a; Brannon et al., 2008). The spatial nature of the variables of interest, the audiovisual nature of the stimuli, and the gross scalp topography of the effects are at least equally consistent with a parietal rather than a temporal–ventral brain origin. This speculation, however, should be followed up using imaging techniques with better spatial resolution if our claim is to be empirically substantiated. Although the waveform morphology of the mid-latency anterior processing was different, timing and topography were consistent with the infant Nc component identified by other as a marker of attentional orienting (see de Haan et al., 2003 or Reynolds and Richards, 2005 for reviews). Morphological differences were likely due to the multimodal nature of our stimuli compared to previous visual work. Previous work has shown that the infant Nc response is related to behavioral attention (e.g. Richards, 2003; also see Reynolds et al., 2010). Unfortunately, behavioral looking times could not be obtained in our study to compare directly with previous behavioral work, given the necessity for presenting many trials of short duration for the ERP paradigm. We were also unable to obtain enough artifact-free trials per stimulus type to look at differences in learning between specific pairings of “long” and “short” auditory and visual stimuli. Nevertheless, “long” and “short” visual and auditory stimuli differed by a 1:3 ratio and previous work suggests a 1:3 difference is well within the 5-month old infants’ ability to discriminate (limit of about a 1:2 ratio; Brannon et al., 2007; Lipton and Spelke, 2003; Wood and Spelke, 2005). Furthermore, the patterns of Nc modulation observed in our study accord with previous behavioral work showing discrimination for space–time pairs when first familiarized to congruent pairings but not when first familiarized to incongruent pairings (Srinivasan and Carey, 2010). We also observed an interaction between Training Condition and Test Type in the late positive slow wave component. Post hoc analysis revealed that only familiar test stimuli for those infants training on incongruent spatial–temporal pairings elicited a late PSW significantly different from baseline. In contrast, neither the familiar or novel test conditions for infants who trained on congruent pairings nor the novel condition for infants trained on incongruent pairings were significantly different from baseline. Previous studies suggest that late positive slow wave (PSW) activity is related to memory updating for partially encoded stimuli, and that stimuli that are fully encoded in memory and stimuli that are not encoded at all do not evoke late slow wave activity above baseline (e.g. de Haan, 2007 or Reynolds and Richards, 2005 for reviews). Although speculative, the return to baseline for the familiar stimuli of those trained with congruent pairings may be a result of the cross-modal spatio-temporal relations being Author's personal copy D.C. Hyde et al. / Developmental Cognitive Neuroscience 6 (2013) 102–112 fully encoded into memory while the lack of a significant PSW for the other conditions may be an indicative of a lack of higher-level encoding at all. In contrast, an increase in amplitude for the familiar test stimuli of those in the incongruent training group may indicate updating of a partially encoded stimulus in memory. More generally, in multisensory contexts, redundancy of information across the senses has been shown to facilitate learning and early development, particularly in young infants around the same age as we test here (see Bahrick and Lickliter, 2000). While onset synchrony may have equally queued training groups into the events, intersensory redundancy cannot explain the differential learning patterns between the training groups of our study because the test stimuli were the same for both groups and because the relational information was not redundant across the senses. While redundancy of temporal information would likely facilitate learning in this context, as it has shown to do in others (e.g. Jordan et al., 2008) and may have in Srinivasan and Carey (2010), our study shows that it is not necessary for learning pairings of audio-visual magnitude information. A relational congruence bias, in addition to intersensory redundancy, should now be considered a cue that facilitates processing of complex audio-visual information. 5. Conclusions In sum, our results suggest that young infants encode, attend to, and potentially remember spatial and temporal relations that are congruent more readily than they do relations that are incongruent. The age of our participants, 5-months, makes it unlikely that explicit instruction or language experience can account for the results. Rather, these results suggest the structure of the mind biases us to learn certain magnitude pairings over others. A remaining issue is if this interaction can be explained by a privileged connection between domain-specific representations of space and time or by generalized representations of magnitude (see Lourenco and Longo, 2011 or Walsh, 2003 for reviews). Another remaining issue is why this bias is present. Is this a feature or a bug of the brain? Like intersensory redundancy (Bahrick and Lickliter, 2000), this bias may further highlight and facilitate learning of important magnitude correspondences that transcend the primary properties of sound and sight early in development. Alternatively, relational congruence bias may result from a recycling of ancient mechanisms originally evolved for other purposes (Gould and Vrba, 1982; Dehaene, 2005). Whatever the answer, it seems as if this organization has important implications for human nature and culture as pervasive as guiding the way we learn about, understand, and choose to describe the world around us (Clark, 1973; Gruber, 1965; Jackendoff, 1983; Lakoff and Johnson, 1980; Langacker, 1987; Srinivasan and Carey, 2010; Talmy, 1988). Conflicts of interest We declare no conflicts of interest in conducting or publishing this work. 111 Acknowledgements This study was funded in part through the support of the Family Studies Center at Brigham Young University and by the Camilla Eyring Kimball Endowment in the College of Family, Home, and Social Sciences at Brigham Young University. References Algom, D., Dekel, A., Pansky, A., 1996. The perception of number from the separability of the stimulus: the Stroop effect revisited. Memory & Cognition 24 (5), 557–572. Bahrick, L.E., Lickliter, R., 2000. Intersensory redundancy guides attentional selectivity and perceptual learning in infancy. Developmental Psychology 36 (2), 190–201. Bahrick, L.E., Lickliter, R., Flom, R., 2004. Intersensory redundancy guides the development of selective attention, perception, and cognition in infancy. Current Directions in Psychological Science 13 (3), 99–102. Boroditsky, L., 2000. Metaphoric structuring: understanding time through spatial metaphors. Cognition 75 (1), 1–28. Boroditsky, L., 2011. How languages construct time. In: Dehaene, S., Brannon, E. (Eds.), Space, Time, and Number in the Brain: Searching for the Foundations of Mathematical Thought. Elsevier, pp. 333–341. Brannon, E.M., Libertus, M., Meck, W.H., Woldorff, M., 2008. Electrophysiological measures of time processing in infants and adults brains: Weber’s law holds. Journal of Cognitive Neuroscience 20 (2), 192–203. Brannon, E.M., Lutz, D., Cordes, S., 2006. The development of area discrimination and its implications for numerical abilities in infancy. Developmental Science 9 (6), 59–64. Brannon, E.M., Suanda, S.H., Libertus, K., 2007. Increasing precision in temporal discrimination over development parallels the development of number discrimination. Developmental Science 10 (6), 770–777. Cantlon, J.F., Platt, M., Brannon, E.M., 2009. Beyond the number domain. Trends in Cognitive Sciences 13 (2), 83–91. Carver, L.J., Dawson, G., Panagiotides, H., Meltzoff, A.N., McPartland, J., Gray, J., Munson, J., 2003. Age-related differences in neural correlates of face recognition during the toddler and preschool years. Developmental Psychobiology 42, 145–159. Casasanto, D., Boroditsky, L., 2008. Time in the mind: using space to think about time. Cognition 106, 579–593. Clark, H.H., 1973. Space, time, semantics and the child. In: Moore, T.E. (Ed.), Cognitive Development and the Acquisition of Language. Academic Press, New York, pp. 27–63. Cohen Kadosh, R., Henik, A., 2006. A common representation for semantic and physical properties: a cognitive-anatomical approach. Journal of Experimental Psychology 53, 87–94. Cohen Kadosh, R., Cohen Kadosh, K., Linden, D.E., Gevers, W., Berger, A., Henik, A., 2007. The brain locus of interaction between number and size: a combined functional magnetic resonance imaging and event-related potential study. Journal of Cognitive Neuroscience 19 (6), 957–970. de Haan, M. (Ed.), 2007. Infant EEG and Event-Related Potentials. Psychology Press, London. de Haan, M., Johnson, M.H., Halit, H., 2003. Development of face-sensitive event-related potential components during infancy. International Journal of Psychophysiology 51, 45–58. de Haan, M., Nelson, C.A., 1997. Recognition of the mother’s face by sixmonth-old infants: a neurobehavioral study. Child Development 68, 187–210. Dehaene, S., 2005. Evolution of human cortical circuits for reading and arithmetic: the neuronal recycling hypothesis. In: Dehaene, S., Duhamel, J.-R., Hauser, M.D., Rizzolatti, G. (Eds.), From Monkey Brain to Human Brain. A Fyssen Foundation Symposium. MIT Press, Cambridge, MA, pp. 133–157. Dehaene, S., Brannon, E.M., 2010. Space, time, and number: a Kantian research program. Trends in Cognitive Sciences 14, 517–519. deHevia, M.D., Spelke, E.S., 2010. Number-space mapping in human infants. Psychological Science 21, 653–660. Droit-Volet, S., Clément, A., Fayol, M., 2008. Time, number and length: similarities and differences in discrimination in adults and children. Quarterly Journal of Experimental Psychology 61 (12), 1827–1846. Duffy, S., Huttenlocher, J., Levine, S.C., 2005. How infants encode spatial extent? Infancy 8 (1), 81–90. Feigenson, L., 2007. The equality of quantity. Trends in Cognitive Sciences 11 (5), 185–187. Author's personal copy 112 D.C. Hyde et al. / Developmental Cognitive Neuroscience 6 (2013) 102–112 Feigenson, L., Dehaene, S., Spelke, E.S., 2004. Core systems of number. Trends in Cognitive Sciences 8 (10), 307–314. Flom, R., Bahrick, L.E., 2010. The effects of intersensory redundancy on attention and memory: infants’ long-term memory for orientation in audiovisual events. Developmental Psychology 46, 428–436. Fias, W., Lammertyn, J., Reynvoet, B., Dupont, P., Orban, G.A., 2003. Parietal representation of symbolic and nonsymbolic magnitude. Journal of Cognitive Neuroscience 15 (1), 47–56. Gelman, R., Williams, E., 1998. Enabling constraints for cognitive development and learning: domain specificity and epigenesis. In: Kuhn, D., Siegler, R. (Eds.), Cognition, Perception and Language. Handbook of Child Psychology, vol. 2, fifth ed. John Wiley & Sons, New York, pp. 575–630. Gould, S., Vrba, E., 1982. Exaptation—a missing term in the science of form. Paleobiology 8, 4–15. Grossmann, T., Gliga, T., Johnson, M.H., Mareschal, D., 2009. The neural basis of perceptual category learning in human infants. Journal of Cognitive Neuroscience 21 (12), 2276–2286. Gruber, J., 1965. Studies in Lexical Relations. MIT, Cambridge. Halberda, J., Feigenson, L., 2008. Developmental change in the acuity of the “number sense”: the approximate number system in 3-, 4-, 5-, and 6-year-olds and adults. Developmental Psychology 44 (5), 1457–1465. Halit, H., Csibra, G., Volein, A., Johnson, M.H., 2004. Face-sensitive cortical processing in early infancy. Journal of Child Psychology and Psychiatry 45, 1228–1234. Hyde, D.C., Boas, D., Blair, C., Carey, S., 2010a. Near-infrared spectroscopy shows right parietal specialization for number in pre-verbal infants. Neuroimage 53 (2), 647–652. Hyde, D.C., Jones, B.L., Flom, R., Porter, C.L., 2011. Neural signatures of face–voice synchrony in 5-month-old human infants. Developmental Psychobiology 53 (4), 359–370. Hyde, D.C., Jones, B.L., Porter, C.L., Flom, R., 2010b. Visual stimulation enhances auditory processing in 3-month-old infants and adults. Developmental Psychobiology 52 (2), 181–189. Hyde, D.C., Spelke, E.S., 2011. Neural signatures of number processing in human infants: evidence for two core systems underlying numerical cognition. Developmental Science, 360–371. Izard, V., Dehaene-Lambertz, G., Dehaene, S., 2008. Distinct cerebral pathways for object identity and number in human infants. PLoS Biology 6 (2), 0275–0285. Izard, V., Sann, C., Spelke, E.S., Streri, A., 2009. Newborn infants perceive abstract numbers. Proceedings of the National Academy of Sciences of the United States of America 106 (June (25)), 10382–10385. Jackendoff, R., 1983. Semantics and Cognition. MIT Press, Cambridge. Johnson, S.P., 2003. The nature of cognitive development. Trends in Cognitive Sciences 7, 102–104. Jordan, K.E., Suanda, S.H., Brannon, E.M., 2008. Intersensory redundancy accelerates preverbal numerical competence. Cognition 108 (1), 210–221. Kant, I., 1998. In: Guyer, P., Wood, A.W. (Trans. & Eds.), Critique of Pure Reason. Cambridge University Press, Cambridge (Original work published 1781). Lakoff, G., Johnson, M., 1980. Metaphors We Live By. University of Chicago Press, Chicago. Langacker, R.W., 1987. Foundations of Cognitive Grammar: Theoretical Prerequisites. Stanford University Press, Stanford, CA. Lipton, J.S., Spelke, E.S., 2003. Origins of number sense: large number discrimination in human infants. Psychological Science 14 (5), 396–401. Lourenco, S.F., Longo, M.R., 2010. General magnitude representations in human infants. Psychological Science 21 (6), 873–881. Lourenco, S.F., Longo, M.R., 2011. Origins and the development of generalized magnitude representation. In: Dehaene, S., Brannon, E. (Eds.), Space, Time, and Number in the Brain: Searching for the Foundations of Mathematical Thought. Elsevier, pp. 225–244. Meck, W.H., Church, R.M., 1983. A mode control model of counting and timing processes. Journal of Experimental Psychology: Animal Behavior Processes 9, 320–334. Mix, K., Huttenlocher, J., Levine, S.C., 2002. Multiple cues for quantification in infancy: is number one of them? Psychological Bulletin 128 (2), 278–294. Mix, K.S., Levine, S.C., Huttenlocher, J., 1999. Early fraction calculation ability. Developmental Psychology 35, 164–174. Nelson, C.A., 1994. Neural correlates of recognition memory in the first postnatal year of life. In: Dawson, G., Fischer, K. (Eds.), Human Development and the Developing Brain. Guilford Press, New York, pp. 269–313. Nelson, C.A., Collins, P.F., 1992. Neural and behavioral correlates of recognition memory in 4- and 8-month-old infants. Brain and Cognition 19, 105–121. Nelson, C.A., de Regnier, R.A., 1992. Neural correlates of attention and memory in the first year of life. Developmental Neuropsychology 8, 119–134. Piaget, J., 1927/1969. The Child’s Conception of Time. Ballantine Books, New York. Pinel, P., Piazza, M., Le Bihan, D., Dehaene, S., 2004. Distributed and overlapping cerebral representations of number size and luminance during comparative judgments. Neuron 41 (6), 983–993. Pinker, S., 1997. How the Mind Works. Norton, New York. Quinn, P.C., Westerlund, A., Nelson, C.A., 2006. Neural markers of categorization in 6 month old infants. Psychological Science 17, 59–66. Reynolds, G.D., Richards, J.E., 2005. Familiarization, attention, and recognition memory in infancy: an event-related potential and cortical source localization study. Developmental Psychology 41, 598–615. Reynolds, G.D., Courage, M.L., Richards, J.E., 2010. Infant attention and visual preferences: converging evidence from behavior, event-related potentials, and cortical source localization. Developmental Psychology 46 (4), 886–904. Richards, J.E., 2003. Attention affects the recognition of briefly presented visual stimuli in infants: an ERP study. Developmental Science 6, 312–328. Scott, L.S., 2011. Mechanisms underlying the emergence of object representations during infancy. Journal of Cognitive Neuroscience 23, 2935–2955. Scott, L.S., Nelson, C.A., 2006. Featural and configural face processing in adults and Infants: a behavioral and electrophysiological Investigation. Perception 35 (8), 1107–1128. Srinivasan, M., Carey, S., 2010. The long and the short of it: on the nature and origin of functional overlap between representations of space and time. Cognition 116 (2), 217–241. Talmy, L., 1988. Force dynamics in language and cognition. Cognitive Science 12, 49–100. Tzelgov, J., Meyer, J., Henik, A., 1992. Automatic and intentional processing of numerical information. Journal of Experimental Psychology: Learning, Memory, and Cognition 18 (1), 166–179. van Marle, K., Wynn, K., 2006. Six-month old infants use analog magnitudes to represent duration. Developmental Science 9 (5), F41–F49. Walsh, V., 2003. A theory of magnitude: common cortical metrics of time, space and quantity. Trends in Cognitive Sciences 7, 483–488. Wood, J.N., Spelke, E.S., 2005. Infants’ enumeration of actions: numerical discrimination and its signature limits. Developmental Science 8 (2), 173–181.
© Copyright 2026 Paperzz