150 Psychologie
Refine
Year of publication
Document Type
- Preprint (112) (remove)
Has Fulltext
- yes (112)
Is part of the Bibliography
- no (112) (remove)
Keywords
- Deutschland (2)
- collaboration script (2)
- natural scenes (2)
- neuronal populations (2)
- primary visual cortex (2)
- referential communication (2)
- stimulus encoding (2)
- visual attention (2)
- Adjustment (1)
- Adulthood (1)
- Anpassung (1)
- Aufsatzsammlung (1)
- College Students (1)
- College Teachers (1)
- Computational model (1)
- Computer Mediated Communication (1)
- Computervermittelte Kommunikation (1)
- Cortical column (1)
- Die kluge Bauerntochter (1)
- Erwachsenenalter (1)
- Frankfurt am Main (1)
- Germany (1)
- Grimm, Jacob (1)
- Grimm, Wilhelm (1)
- Hochschuldidaktik (1)
- Hochschullehrer (1)
- Hypercolumn (1)
- Language (1)
- Lehrer-Schüler-Interaktion (1)
- Lehrerbildung (1)
- Lehrermerkmale (1)
- Liebesbrief (1)
- Märchen (1)
- Neural map (1)
- Object recognition (1)
- Optimal wiring (1)
- Orientation preference (1)
- Pinwheel (1)
- Praxis (1)
- Scene context effects (1)
- Schüler- und Studierendeneinstellungen (1)
- Sprache (1)
- Student Attitudes (1)
- Studentenschaft (1)
- Studienverhalten (1)
- Studierende (Anfangssemester) (1)
- Studium (1)
- Teacher Characteristics (1)
- Teacher Student Interaction (1)
- Tiefenpsychologie (1)
- Viewpoint dependence (1)
- Visual cortex (1)
- Zweierbeziehung (1)
- change blindness (1)
- cognitive demands (1)
- episodic memory (1)
- functional connectivity (1)
- general cognitive ability (1)
- machine learning (1)
- prediction error (1)
- predictive modelling (1)
- predictive processing (1)
- remote collaboration (1)
- resting state (1)
- shared application (1)
- time pressure (1)
Institute
- Ernst Strüngmann Institut (48)
- Psychologie (44)
- MPI für Hirnforschung (32)
- Frankfurt Institute for Advanced Studies (FIAS) (22)
- Medizin (10)
- MPI für empirische Ästhetik (7)
- Starker Start ins Studium: Qualitätspakt Lehre (5)
- Informatik (3)
- Biowissenschaften (1)
- Deutsches Institut für Internationale Pädagogische Forschung (DIPF) (1)
Generating predictions about environmental regularities, relying on these predictions, and updating these predictions when there is a violation from incoming sensory evidence are considered crucial functions of our cognitive system for being adaptive in the future. The violation of a prediction can result in a prediction error (PE) which affects subsequent memory processing. In our preregistered studies, we examined the effects of different levels of PE on episodic memory. Participants were asked to generate predictions about the associations between sequentially presented cue-target pairs, which were violated later with individual items in three PE levels, namely low, medium, and high PE. Hereafter, participants were asked to provide old/new judgments on the items with confidence ratings, and to retrieve the paired cues. Our results indicated a better recognition memory for low PE than medium and high PE levels, suggesting a memory congruency effect. On the other hand, there was no evidence of memory benefit for high PE level. Together, these novel and coherent findings strongly suggest that high PE does not guarantee better memory.
From early to middle childhood, brain regions that underlie memory consolidation undergo profound maturational changes. However, there is little empirical investigation that directly relates age-related differences in brain structural measures to the memory consolidation processes. The present study examined system-level memory consolidations of intentionally studied object-location associations after one night of sleep (short delay) and after two weeks (long delay) in normally developing 5-to-7-year-old children (n = 50) and young adults (n = 39). Behavioural differences in memory consolidation were related to structural brain measures. Our results showed that children, in comparison to young adults, consolidate correctly learnt object-location associations less robustly over short and long delay. Moreover, using partial least squares correlation method, a unique multivariate profile comprised of specific neocortical (prefrontal, parietal, and occipital), cerebellar, and hippocampal subfield structures was found to be associated with variation in short-delay memory consolidation. A different multivariate profile comprised of a reduced set of brain structures, mainly consisting of neocortical (prefrontal, parietal, and occipital), and selective hippocampal subfield structures (CA1-2 and subiculum) was associated with variation in long-delay memory consolidation. Taken together, the results suggest that multivariate structural pattern of unique sets of brain regions are related to variations in short- and long-delay memory consolidation across children and young adults.
RESEARCH HIGHLIGHTS
* Short- and long-delay memory consolidation is less robust in children than in young adults
* Short-delay brain profile comprised of hippocampal, cerebellar, and neocortical brain regions
* Long-delay brain profile comprised of neocortical and selected hippocampal brain regions.
* Brain profiles differ between children and young adults.
From early to middle childhood, brain regions that underlie memory consolidation undergo profound maturational changes. However, there is little empirical investigation that directly relates age-related differences in brain structural measures to the memory consolidation processes. The present study examined system-level memory consolidations of intentionally studied object-location associations after one night of sleep (short delay) and after two weeks (long delay) in normally developing 5-to-7-year-old children (n = 50) and young adults (n = 39). Behavioural differences in memory consolidation were related to structural brain measures. Our results showed that children, in comparison to young adults, consolidate correctly learnt object-location associations less robustly over short and long delay. Moreover, using partial least squares correlation method, a unique multivariate profile comprised of specific neocortical (prefrontal, parietal, and occipital), cerebellar, and hippocampal subfield structures was found to be associated with variation in short-delay memory consolidation. A different multivariate profile comprised of a reduced set of brain structures, mainly consisting of neocortical (prefrontal, parietal, and occipital), and selective hippocampal subfield structures (CA1-2 and subiculum) was associated with variation in long-delay memory consolidation. Taken together, the results suggest that multivariate structural pattern of unique sets of brain regions are related to variations in short- and long-delay memory consolidation across children and young adults.
RESEARCH HIGHLIGHTS
Short- and long-delay memory consolidation is less robust in children than in young adults
* Short-delay brain profile comprised of hippocampal, cerebellar, and neocortical brain regions
* Long-delay brain profile comprised of neocortical and selected hippocampal brain regions.
* Brain profiles differ between children and young adults.
Memory consolidation tends to be less robust in childhood than adulthood. However, little is known about the corresponding functional differences in the developing brain that may underlie age-related differences in retention of memories over time. This study examined system-level memory consolidation of object-scene associations after learning (immediate delay), one night of sleep (short delay), as well as two weeks (long delay) in 5-to-7-year-old children (n = 49) and in young adults (n = 39), as a reference group with mature consolidation systems. Particularly, we characterized how functional neural activation and reinstatement of neural patterns change over time, assessed by functional magnetic resonance imaging combined with representational similarity analysis (RSA). Our results showed that memory consolidation in children was less robust and strong (i.e., more forgetting) compared to young adults. Contrasting correctly retained remote versus recent memories across time delay, children showed less upregulation in posterior parahippocampal gyrus, lateral occipital cortex, and cerebellum than adults. In addition, both children and adults showed decrease in scene-specific neural reinstatement over time, indicating time-related decay of detailed differentiated memories. At the same time, we observed more generic gist-like neural reinstatement in medial-temporal and prefrontal brain regions uniquely in children, indicating qualitative difference in memory trace in children. Taken together, 5-to-7-year-old children, compared to young adults, show less robust memory consolidation, possibly due to difficulties in engaging in differentiated neural reinstatement in neocortical mnemonic regions during retrieval of remote memories, coupled with relying more on gist-like generic neural reinstatement.
Memory consolidation tends to be less robust in childhood than adulthood. However, little is known about the corresponding functional differences in the developing brain that may underlie age-related differences in retention of memories over time. This study examined system-level memory consolidation of object-scene associations after learning (immediate delay), one night of sleep (short delay), as well as two weeks (long delay) in 5-to-7-year-old children (n = 49) and in young adults (n = 39), as a reference group with mature consolidation systems. Particularly, we characterized how functional neural activation and reinstatement of neural patterns change over time, assessed by functional magnetic resonance imaging combined with representational (dis)similarity analysis (RSA). Our results showed that memory consolidation in children was less robust (i.e., more forgetting) compared to young adults. For correctly retained remote memories, young adults showed increased neural activation from short to long delay in neocortical (parietal, prefrontal and occipital) and cerebellar brain regions, while children showed increased neural activation in prefrontal and decrease in neural activity in parietal brain regions over time. In addition, there was an overall attenuated scene-specific memory reinstatement of neural patterns in children compared to young adults. At the same time, we observed category-based reinstatement in medial-temporal, neocortical (prefrontal and parietal), and cerebellar brain regions only in children. Taken together, 5-to-7-year-old children, compared to young adults, show less robust memory consolidation, possibly due to difficulties in engaging in differentiated neural reinstatement in neocortical mnemonic regions during retrieval of remote memories, coupled with relying more on gist-like, category-based neural reinstatement.
Efficient processing of visual environment necessitates the integration of incoming sensory evidence with concurrent contextual inputs and mnemonic content from our past experiences. To delineate how this integration takes place in the brain, we studied modulations of feedback neural patterns in non-stimulated areas of the early visual cortex in humans (i.e., V1 and V2). Using functional magnetic resonance imaging and multivariate pattern analysis, we show that both, concurrent contextual and time-distant mnemonic information, coexist in V1/V2 as feedback signals. The extent to which mnemonic information is reinstated in V1/V2 depends on whether the information is retrieved episodically or semantically. These results demonstrate that our stream of visual experience contains not just information from the visual surrounding, but also memory-based predictions internally generated in the brain.
Human functional brain connectivity can be temporally decomposed into states of high and low cofluctuation, defined as coactivation of brain regions over time. Rare states of particularly high cofluctuation have been shown to reflect fundamentals of intrinsic functional network architecture and to be highly subject-specific. However, it is unclear whether such network-defining states also contribute to individual variations in cognitive abilities – which strongly rely on the interactions among distributed brain regions. By introducing CMEP, a new eigenvector-based prediction framework, we show that as few as 16 temporally separated time frames (< 1.5% of 10min resting-state fMRI) can significantly predict individual differences in intelligence (N = 263, p < .001). Against previous expectations, individual’s network-defining time frames of particularly high cofluctuation do not predict intelligence. Multiple functional brain networks contribute to the prediction, and all results replicate in an independent sample (N = 831). Our results suggest that although fundamentals of person-specific functional connectomes can be derived from few time frames of highest connectivity, temporally distributed information is necessary to extract information about cognitive abilities. This information is not restricted to specific connectivity states, like network-defining high-cofluctuation states, but rather reflected across the entire length of the brain connectivity time series.
Studying the neural basis of human dynamic visual perception requires extensive experimental data to evaluate the large swathes of functionally diverse brain neural networks driven by perceiving visual events. Here, we introduce the BOLD Moments Dataset (BMD), a repository of whole-brain fMRI responses to over 1,000 short (3s) naturalistic video clips of visual events across ten human subjects. We use the videos’ extensive metadata to show how the brain represents word- and sentence-level descriptions of visual events and identify correlates of video memorability scores extending into the parietal cortex. Furthermore, we reveal a match in hierarchical processing between cortical regions of interest and video-computable deep neural networks, and we showcase that BMD successfully captures temporal dynamics of visual events at second resolution. With its rich metadata, BMD offers new perspectives and accelerates research on the human brain basis of visual event perception.
Understanding effects of emotional valence and stress on children’s memory is important for educational and legal contexts. This study disentangles the effects of emotional content of to-be-remembered information (i.e., items differing in emotional valence and arousal), stress exposure, and associated cortisol secretion on children’s memory. We also examine whether girls’ memory is more affected by stress induction. 143 6-to-7-year-old children were randomly allocated to the Trier Social Stress Test for Children (n = 103) or a control condition (n = 40). 25 minutes after stressor onset, children incidentally encoded 75 objects varying in emotional valence (crossed with arousal) together with neutral scene backgrounds. We found that response-bias corrected memory was worse for low arousing negative items than neutral and positive items, with the latter two categories not being different from each other. Whilst boys’ memory was largely unaffected by stress, girls in the stress condition showed worse memory for negative items, especially the low arousing ones, than girls in the control condition. Girls, compared to boys, reported higher subjective stress increases following stress exposure, and had higher cortisol stress responses. Whilst a higher cortisol stress response was associated with better emotional memory in girls in the stress condition, boys’ memory was not associated with their cortisol secretion. Taken together, our study suggests that 6-to-7-year-old children, more so girls, show memory suppression for negative information. Girls’ memory for negative information, compared to boys, is also more strongly modulated by stress experience and the associated cortisol response.
Rhythmic neural spiking and attentional sampling arising from cortical receptive field interactions
(2018)
Summary: Growing evidence suggests that distributed spatial attention may invoke theta (3-9 Hz) rhythmic sampling processes. The neuronal basis of such attentional sampling is however not fully understood. Here we show using array recordings in visual cortical area V4 of two awake macaques that presenting separate visual stimuli to the excitatory center and suppressive surround of neuronal receptive fields elicits rhythmic multi-unit activity (MUA) at 3-6 Hz. This neuronal rhythm did not depend on small fixational eye movements. In the context of a distributed spatial attention task, during which the monkeys detected a spatially and temporally uncertain target, reaction times (RT) exhibited similar rhythmic fluctuations. RTs were fast or slow depending on the target occurrence during high or low MUA, resulting in rhythmic MUA-RT cross-correlations at at theta frequencies. These findings suggest that theta-rhythmic neuronal activity arises from competitive receptive field interactions and that this rhythm may subserve attentional sampling.
Highlights:
* Center-surround interactions induce theta-rhythmic MUA of visual cortex neurons
* The MUA rhythm does not depend on small fixational eye movements
* Reaction time fluctuations lock to the neuronal rhythm under distributed attention
Can prediction error explain predictability effects on the N1 during picture-word verification?
(2024)
Do early effects of predictability in visual word recognition reflect prediction error? Electrophysiological research investigating word processing has demonstrated predictability effects in the N1, or first negative component of the event-related potential (ERP). However, findings regarding the magnitude of effects and potential interactions of predictability with lexical variables have been inconsistent. Moreover, past studies have typically used categorical designs with relatively small samples and relied on by-participant analyses. Nevertheless, reports have generally shown that predicted words elicit less negative-going (i.e., lower amplitude) N1s, a pattern consistent with a simple predictive coding account. In our preregistered study, we tested this account via the interaction between prediction magnitude and certainty. A picture-word verification paradigm was implemented in which pictures were followed by tightly matched picture-congruent or picture-incongruent written nouns. The predictability of target (picture-congruent) nouns was manipulated continuously based on norms of association between a picture and its name. ERPs from 68 participants revealed a pattern of effects opposite to that expected under a simple predictive coding framework.
The hippocampal-dependent memory system and striatal-dependent memory system modulate reinforcement learning depending on feedback timing in adults, but their contributions during development remain unclear. In a 2-year longitudinal study, 6-to-7-year-old children performed a reinforcement learning task in which they received feedback immediately or with a short delay following their response. Children’s learning was found to be sensitive to feedback timing modulations in their reaction time and inverse temperature parameter, which quantifies value-guided decision-making. They showed longitudinal improvements towards more optimal value-based learning, and their hippocampal volume showed protracted maturation. Better delayed model-derived learning covaried with larger hippocampal volume longitudinally, in line with the adult literature. In contrast, a larger striatal volume in children was associated with both better immediate and delayed model-derived learning longitudinally. These findings show, for the first time, an early hippocampal contribution to the dynamic development of reinforcement learning in middle childhood, with neurally less differentiated and more cooperative memory systems than in adults.
The hippocampal-dependent memory system and striatal-dependent memory system modulate reinforcement learning depending on feedback timing in adults, but their contributions during development remain unclear. In a 2-year longitudinal study, 6-to-7-year-old children performed a reinforcement learning task in which they received feedback immediately or with a short delay following their response. Children’s learning was found to be sensitive to feedback timing modulations in their reaction time and inverse temperature parameter, which quantifies value-guided decision-making. They showed longitudinal improvements towards more optimal value-based learning, and their hippocampal volume showed protracted maturation. Better delayed model-derived learning covaried with larger hippocampal volume longitudinally, in line with the adult literature. In contrast, a larger striatal volume in children was associated with both better immediate and delayed model-derived learning longitudinally. These findings show, for the first time, an early hippocampal contribution to the dynamic development of reinforcement learning in middle childhood, with neurally less differentiated and more cooperative memory systems than in adults.
The hippocampal-dependent memory system and striatal-dependent memory system modulate reinforcement learning depending on feedback timing in adults, but their contributions during development remain unclear. In a 2-year longitudinal study, 6-to-7-year-old children performed a reinforcement learning task in which they received feedback immediately or with a short delay following their response. Children’s learning was found to be sensitive to feedback timing modulations in their reaction time and inverse temperature parameter, which quantifies value-guided decision-making. They showed longitudinal improvements towards more optimal value-based learning, and their hippocampal volume showed protracted maturation. Better delayed model-derived learning covaried with larger hippocampal volume longitudinally, in line with the adult literature. In contrast, a larger striatal volume in children was associated with both better immediate and delayed model-derived learning longitudinally. These findings show, for the first time, an early hippocampal contribution to the dynamic development of reinforcement learning in middle childhood, with neurally less differentiated and more cooperative memory systems than in adults.
The hippocampal-dependent memory system and striatal-dependent memory system modulate reinforcement learning depending on feedback timing in adults, but their contributions during development remain unclear. In a 2-year longitudinal study, 6-to-7-year-old children performed a reinforcement learning task in which they received feedback immediately or with a short delay following their response. Children’s learning was found to be sensitive to feedback timing modulations in their reaction time and inverse temperature parameter, which quantifies value-guided decision-making. They showed longitudinal improvements towards more optimal value-based learning, and their hippocampal volume showed protracted maturation. Better delayed model-derived learning covaried with larger hippocampal volume longitudinally, in line with the adult literature. In contrast, a larger striatal volume in children was associated with both better immediate and delayed model-derived learning longitudinally. These findings show, for the first time, an early hippocampal contribution to the dynamic development of reinforcement learning in middle childhood, with neurally less differentiated and more cooperative memory systems than in adults.
Metacognition plays a pivotal role in human development. The ability to realize that we do not know something, or meta-ignorance, emerges after approximately five years of age. We aimed at identifying the brain systems that underlie the developmental emergence of this ability in a preschool sample.
Twenty-four children aged between five and six years answered questions under three conditions of a meta-ignorance task twice. In the critical partial knowledge condition, an experimenter first showed two toys to a child, then announced that she would place one of them in a box behind a screen, out of sight from the child. The experimenter then asked the child whether or not she knew which toy was in the box.
Children who answered correctly both times to the metacognitive question in the partial knowledge condition (n=9) showed greater cortical thickness in a cluster within left medial orbitofrontal cortex than children who did not (n=15). Further, seed-based functional connectivity analyses of the brain during resting state revealed that this region is functionally connected to the medial orbitofrontal gyrus, posterior cingulate gyrus and precuneus, and mid- and inferior temporal gyri.
This finding suggests that the default mode network, critically through its prefrontal regions, supports introspective processing. It leads to the emergence of metacognitive monitoring allowing children to explicitly report their own ignorance.
Spontaneous brain activity builds the foundation for human cognitive processing during external demands. Neuroimaging studies based on functional magnetic resonance imaging (fMRI) identified specific characteristics of spontaneous (intrinsic) brain dynamics to be associated with individual differences in general cognitive ability, i.e., intelligence. However, fMRI research is inherently limited by low temporal resolution, thus, preventing conclusions about neural fluctuations within the range of milliseconds. Here, we used resting-state electroencephalographical (EEG) recordings from 144 healthy adults to test whether individual differences in intelligence (Raven’s Advanced Progressive Matrices scores) can be predicted from the complexity of temporally highly resolved intrinsic brain signals. We compared different operationalizations of brain signal complexity (multiscale entropy, Shannon entropy, Fuzzy entropy, and specific characteristics of microstates) regarding their relation to intelligence. The results indicate that associations between brain signal complexity measures and intelligence are of small effect sizes (r ~ .20) and vary across different spatial and temporal scales. Specifically, higher intelligence scores were associated with lower complexity in local aspects of neural processing, and less activity in task-negative brain regions belonging to the defaultmode network. Finally, we combined multiple measures of brain signal complexity to show that individual intelligence scores can be significantly predicted with a multimodal model within the sample (10-fold cross-validation) as well as in an independent sample (external replication, N = 57). In sum, our results highlight the temporal and spatial dependency of associations between intelligence and intrinsic brain dynamics, proposing multimodal approaches as promising means for future neuroscientific research on complex human traits.
Significance Statement Spontaneous brain activity builds the foundation for intelligent processing - the ability of humans to adapt to various cognitive demands. Using resting-state EEG, we extracted multiple aspects of temporally highly resolved intrinsic brain dynamics to investigate their relationship with individual differences in intelligence. Single associations were of small effect sizes and varied critically across spatial and temporal scales. However, combining multiple measures in a multimodal cross-validated prediction model, allows to significantly predict individual intelligence scores in unseen participants. Our study adds to a growing body of research suggesting that observable associations between complex human traits and neural parameters might be rather small and proposes multimodal prediction approaches as promising tool to derive robust brain-behavior relations despite limited sample sizes.
Adaptive threshold estimation procedures sample close to a subject’s perceptual threshold by dynamically adapting the stimulation based on the subject’s performance. Yet, perceptual thresholds not only depend on the observers’ sensory capabilities but also on any bias in terms of their expectations and response preferences, thus distorting the precision of the threshold estimates. Using the framework of signal detection theory (SDT), independent estimates of both, an observer’s sensitivity and internal processing bias can be delineated from threshold estimates. While this approach is commonly available for estimation procedures engaging the method of constant stimuli (MCS), correction procedures for adaptive methods (AM) are only scarcely applied. In this article, we introduce a new AM that takes individual biases into account, and that allows for a bias-corrected assessment of subjects’ sensitivity. This novel AM is validated with simulations and compared to a typical MCS-procedure, for which the implementation of bias correction has been previously demonstrated.
Comparing AM and MCS demonstrates the viability of the presented AM. Besides its feasibility, the results of the simulation reveal both, advantages, and limitations of the proposed AM. The procedure has considerable practical implications, in particular for the design of shaping procedures in sensory training experiments, in which task difficulty has to be constantly adapted to an observer’s performance, to improve training efficiency.
In a dynamic environment, the already limited information that human working memory can maintain needs to be constantly updated to optimally guide behaviour. Indeed, previous studies showed that working memory representations are continuously being transformed during delay periods leading up to a response. This goes hand-in-hand with the removal of task-irrelevant items. However, does such removal also include veridical, original stimuli, as they were prior to transformation? Here we aimed to assess the neural representation of task-relevant transformed representations, compared to the no-longer-relevant veridical representations they originated from. We applied multivariate pattern analysis to electroencephalographic data during maintenance of orientation gratings with and without mental rotation. During maintenance, we perturbed the representational network by means of a visual impulse stimulus, and were thus able to successfully decode veridical as well as imaginary, transformed orientation gratings from impulse-driven activity. On the one hand, the impulse response reflected only task-relevant (cued), but not task-irrelevant (uncued) items, suggesting that the latter were quickly discarded from working memory. By contrast, even though the original cued orientation gratings were also no longer task-relevant after mental rotation, these items continued to be represented next to the rotated ones, in different representational formats. This seemingly inefficient use of scarce working memory capacity was associated with reduced probe response times and may thus serve to increase precision and flexibility in guiding behaviour in dynamic environments.
We explore the potential of optically-pumped magnetometers (OPMs) to infer the laminar origins of neural activity non-invasively. OPM sensors can be positioned closer to the scalp than conventional cryogenic MEG sensors, opening an avenue to higher spatial resolution when combined with high-precision forward modelling. By simulating the forward model projection of single dipole sources onto OPM sensor arrays with varying sensor densities and measurement axes, and employing sparse source reconstruction approaches, we find that laminar inference with OPM arrays is possible at relatively low sensor counts at moderate to high signal-to-noise ratios (SNR). We observe improvements in laminar inference with increasing spatial sampling densities and number of measurement axes. Surprisingly, moving sensors closer to the scalp is less advantageous than anticipated - and even detrimental at high SNRs. Biases towards both the superficial and deep surfaces at very low SNRs and a notable bias towards the deep surface when combining empirical Bayesian beamformer (EBB) source reconstruction with a whole-brain analysis pose further challenges. Adequate SNR through appropriate trial numbers and shielding, as well as precise co-registration, is crucial for reliable laminar inference with OPMs.
An important question concerning inter-areal communication in the cortex is whether these interactions are synergistic, i.e. brain signals can either share common information (redundancy) or they can encode complementary information that is only available when both signals are considered together (synergy). Here, we dissociated cortical interactions sharing common information from those encoding complementary information during prediction error processing. To this end, we computed co-information, an information-theoretical measure that distinguishes redundant from synergistic information among brain signals. We analyzed auditory and frontal electrocorticography (ECoG) signals in five common awake marmosets performing two distinct auditory oddball tasks and investigated to what extent event-related potentials (ERP) and broadband (BB) dynamics encoded redundant and synergistic information during auditory prediction error processing. In both tasks, we observed multiple patterns of synergy across the entire cortical hierarchy with distinct dynamics. The information conveyed by ERPs and BB signals was highly synergistic even at lower stages of the hierarchy in the auditory cortex, as well as between auditory and frontal regions. Using a brain-constrained neural network, we simulated the spatio-temporal patterns of synergy and redundancy observed in the experimental results and further demonstrated that the emergence of synergy between auditory and frontal regions requires the presence of strong, long-distance, feedback and feedforward connections. These results indicate that the distributed representations of prediction error signals across the cortical hierarchy can be highly synergistic.
An important question concerning inter-areal communication in the cortex is whether these interactions are synergistic, i.e. convey information beyond what can be performed by isolated signals. In other words, any two signals can either share common information (redundancy) or they can encode complementary information that is only available when both signals are considered together (synergy). Here, we dissociated cortical interactions sharing common information from those encoding complementary information during prediction error processing. To this end, we computed co-information, an information-theoretical measure that distinguishes redundant from synergistic information among brain signals. We analyzed auditory and frontal electrocorticography (ECoG) signals in five common awake marmosets performing two distinct auditory oddball tasks, and investigated to what extent event-related potentials (ERP) and broadband (BB) dynamics exhibit redundancy and synergy for auditory prediction error signals. We observed multiple patterns of redundancy and synergy across the entire cortical hierarchy with distinct dynamics. The information conveyed by ERPs and BB signals was highly synergistic even at lower stages of the hierarchy in the auditory cortex, as well as between lower and higher areas in the frontal cortex. These results indicate that the distributed representations of prediction error signals across the cortical hierarchy can be highly synergistic.
An important question concerning inter-areal communication in the cortex, is whether these interactions are synergistic, i.e. convey information beyond what can be performed by isolated signals. Here, we dissociated cortical interactions sharing common information from those encoding complementary information during prediction error processing. To this end, we computed co-information, an information-theoretical measure that distinguishes redundant from synergistic information among brain signals. We analyzed auditory and frontal electrocorticography (ECoG) signals in three common awake marmosets and investigated to what extent event-related-potentials (ERP) and broadband (BB) dynamics exhibit redundancy and synergy in auditory prediction error signals. We observed multiple patterns of redundancy and synergy across the entire cortical hierarchy with distinct dynamics. The information conveyed by ERPs and BB signals was highly synergistic even at lower stages of the hierarchy in the auditory cortex, as well as between lower and higher areas in the frontal cortex. These results indicate that the distributed representations of prediction error signals across the cortical hierarchy can be highly synergistic.
Natural scene responses in the primary visual cortex are modulated simultaneously by attention and by contextual signals about scene statistics stored across the connectivity of the visual processing hierarchy. Here, we hypothesized that attentional and contextual top-down signals interact in V1, in a manner that primarily benefits the representation of natural visual stimuli, rich in high-order statistical structure. Recording from two macaques engaged in a spatial attention task, we found that attention enhanced the decodability of stimulus identity from population responses evoked by natural scenes but, critically, not by synthetic stimuli in which higher-order statistical regularities were eliminated. Population analysis revealed that neuronal responses converged to a low dimensional subspace for natural but not for synthetic images. Critically, we determined that the attentional enhancement in stimulus decodability was captured by the dominant low dimensional subspace, suggesting an alignment between the attentional and natural stimulus variance. The alignment was pronounced for late evoked responses but not for early transient responses of V1 neurons, supporting the notion that top-down feedback was required. We argue that attention and perception share top-down pathways, which mediate hierarchical interactions optimized for natural vision.
Context information supports serial dependence of multiple visual objects across memory episodes
(2019)
Visual perception operates in an object-based manner, by integrating associated features via attention. Working memory allows a flexible access to a limited number of currently relevant objects, even when they are occluded or physically no longer present. Recently, it has been shown that we compensate for small changes of an object’s feature over memory episodes, which can support its perceptual stability. This phenomenon was termed ‘serial dependence’ and has mostly been studied in situations that comprised only a single relevant object. However, since we are typically confronted with situations where several objects have to be perceived and held in working memory, the central question of how we selectively create temporal stability of several objects has remained unsolved. As different objects can be distinguished by their accompanying context features, like their color or temporal position, we tested whether serial dependence is supported by the congruence of context features across memory episodes. Specifically, we asked participants to remember the motion directions of two sequentially presented colored dot fields per trial. At the end of a trial one motion direction was cued for continuous report either by its color (Experiment 1) or serial position (Experiment 2). We observed serial dependence, i.e., an attractive bias of currently toward previously memorized objects, between current and past motion directions that was clearly enhanced when items had the same color or serial position across trials. This bias was particularly pronounced for the context feature that was used for cueing and for the target of the previous trial. Together, these findings demonstrate that coding of current object representations depends on previous representations, especially when they share similar content and context features. Apparently the binding of content and context features is not completely erased after a memory episode, but it is carried over to subsequent episodes. As this reflects temporal dependencies in natural settings, the present findings reveal a mechanism that integrates corresponding bundles of content and context features to support stable representations of individualized objects over time.
We present a model for the autonomous learning of active binocular vision using a recently developed biome-chanical model of the human oculomotor system. The model is formulated in the Active Efficient Coding (AEC) framework, a recent generalization of classic efficient coding theories to active perception. The model simultaneously learns how to efficiently encode binocular images and how to generate accurate vergence eye movements that facilitate efficient encoding of the visual input. In order to resolve the redundancy problem arising from the actuation of the eyes through antagonistic muscle pairs, we consider the metabolic costs associated with eye movements. We show that the model successfully learns to trade off vergence accuracy against the associated metabolic costs, producing high fidelity vergence eye movements obeying Sherrington’s law of reciprocal innervation.
Anticipating future events is a key computational task for neuronal networks. Experimental evidence suggests that reliable temporal sequences in neural activity play a functional role in the association and anticipation of events in time. However, how neurons can differentiate and anticipate multiple spike sequences remains largely unknown. We implement a learning rule based on predictive processing, where neurons exclusively fire for the initial, unpredictable inputs in a spiking sequence, leading to an efficient representation with reduced post-synaptic firing. Combining this mechanism with inhibitory feedback leads to sparse firing in the network, enabling neurons to selectively anticipate different sequences in the input. We demonstrate that intermediate levels of inhibition are optimal to decorrelate neuronal activity and to enable the prediction of future inputs. Notably, each sequence is independently encoded in the sparse, anticipatory firing of the network. Overall, our results demonstrate that the interplay of self-supervised predictive learning rules and inhibitory feedback enables fast and efficient classification of different input sequences.
Representational Similarity Analysis (RSA) is an innovative approach used to compare neural representations across individuals, species and computational models. Despite its popularity within neuroscience, psychology and artificial intelligence, this approach has led to difficult-to-reconcile and contradictory findings, particularly when comparing primate visual representations with deep neural networks (DNNs). Here, we demonstrate how such contradictory findings could arise due to incorrect inferences about mechanism when comparing complex systems processing high-dimensional stimuli. In a series of studies comparing computational models, primate cortex and human cortex we find two problematic phenomena: a “mimic effect”, where confounds in stimuli can lead to high RSA-scores between provably dissimilar systems, and a “modulation effect”, where RSA- scores become dependent on stimuli used for testing. Since our results bear on a number of influential findings, we provide recommendations to avoid these pitfalls and sketch a way forward to a more solid science of representation in cognitive systems.
Some pitfalls of measuring representational similarity using Representational Similarity Analysis
(2022)
A core challenge in cognitive and brain sciences is to assess whether different biological systems represent the world in a similar manner. Representational Similarity Analysis (RSA) is an innovative approach that addresses this problem by looking for a second-order isomorphisim in neural activation patterns. This innovation makes it easy to compare latent representations across individuals, species and computational models, and accounts for its popularity across disciplines ranging from artificial intelligence to computational neuroscience. Despite these successes, using RSA has led to difficult-to-reconcile and contradictory findings, particularly when comparing primate visual representations with deep neural networks (DNNs): even though DNNs have been shown to learn and behave in vastly different ways to humans, comparisons based on RSA have shown striking similarities in some studies. Here, we demonstrate some pitfalls of using RSA and explain how contradictory findings can arise due to false inferences about representational similarity based on RSA-scores. In a series of studies that capture increasingly plausible training and testing scenarios, we compare neural representations in computational models, primate cortex and human cortex. These studies reveal two problematic phenomena that are ubiquitous in current research: a “mimic effect”, where confounds in stimuli can lead to high RSA-scores between provably dissimilar systems, and a “modulation effect”, where RSA-scores become dependent on stimuli used for testing. Since our results bear on a number of influential findings, such as comparisons made between human visual representations and those of primates and DNNs, we provide recommendations to avoid these pitfalls and sketch a way forward to a more solid science of representation in cognitive systems.
The pitfalls of measuring representational similarity using representational similarity analysis
(2022)
A core challenge in cognitive and brain sciences is to assess whether different biological systems represent the world in a similar manner. Representational Similarity Analysis (RSA) is an innovative approach to address this problem and has become increasingly popular across disciplines ranging from artificial intelligence to computational neuroscience. Despite these successes, RSA regularly uncovers difficult-to-reconcile and contradictory findings. Here, we demonstrate the pitfalls of using RSA and explain how contradictory findings arise due to false inferences about representational similarity based on RSA-scores. In a series of studies that capture increasingly plausible training and testing scenarios, we compare neural representations in computational models, primate cortex and human cortex. These studies reveal two problematic phenomena that are ubiquitous in current research: a “mimic” effect, where confounds in stimuli can lead to high RSA-scores between provably dissimilar systems, and a “modulation effect”, where RSA-scores become dependent on stimuli used for testing. Since our results bear on a number of influential findings and the inferences drawn by current practitioners in a wide range of disciplines, we provide recommendations to avoid these pitfalls and sketch a way forward to a more solid science of representation in cognitive systems.
The pitfalls of measuring representational similarity using representational similarity analysis
(2022)
A core challenge in neuroscience is to assess whether diverse systems represent the world similarly. Representational Similarity Analysis (RSA) is an innovative approach to address this problem and has become increasingly popular across disciplines from machine learning to computational neuroscience. Despite these successes, RSA regularly uncovers difficult-to-reconcile and contradictory findings. Here we demonstrate the pitfalls of using RSA to infer representational similarity and explain how contradictory findings arise and support false inferences when left unchecked. By comparing neural representations in primate, human and computational models, we reveal two problematic phenomena that are ubiquitous in current research: a “mimic” effect, where confounds in stimuli can lead to high RSA scores between provably dissimilar systems, and a “modulation effect”, where RSA-scores become dependent on stimuli used for testing. Since our results bear on existing findings and inferences, we provide recommendations to avoid these pitfalls and sketch a way forward.
Objects that are congruent with a scene are recognised more efficiently than objects that are incongruent. Further, semantic integration of incongruent objects elicits a stronger N300/N400 EEG component. Yet, the time course and mechanisms of how contextual information supports access to semantic object information is unclear. We used computational modelling and EEG to test how context influences semantic object processing. Using representational similarity analysis, we established that EEG patterns dissociated between objects in congruent or incongruent scenes from around 300 ms. By modelling semantic processing of objects using independently normed properties, we confirm that the onset of semantic processing of both congruent and incongruent objects is similar (∼150 ms). Critically, after ∼275 ms, we discover a difference in the duration of semantic integration, lasting longer for incongruent compared to congruent objects. These results constrain our understanding of how contextual information supports access to semantic object information.
Objects that are congruent with a scene are recognised more efficiently than objects that are incongruent. Further, semantic integration of incongruent objects elicits a stronger N300/N400 EEG component. Yet, the time course and mechanisms of how contextual information supports access to semantic object information is unclear. We used computational modelling and EEG to test how context influences semantic object processing. Using representational similarity analysis, we established that EEG patterns dissociated between objects in congruent or incongruent scenes from around 300 ms. By modelling semantic processing of objects using independently normed properties, we confirm that the onset of semantic processing of both congruent and incongruent objects is similar (∼150 ms). Critically, after ∼275 ms, we discover a difference in the duration of semantic integration, lasting longer for incongruent compared to congruent objects. These results constrain our understanding of how contextual information supports access to semantic object information.
Attention selects relevant information regardless of whether it is physically present or internally stored in working memory. Perceptual research has shown that attentional selection of external information is better conceived as rhythmic prioritization than as stable allocation. Here we tested this principle using information processing of internal representations held in working memory. Participants memorized four spatial positions that formed the endpoints of two objects. One of the positions was cued for a delayed match-non-match test. When uncued positions were probed, participants responded faster to uncued positions located on the same object as the cued position than to those located on the other object, revealing object-based attention in working memory. Manipulating the interval between cue and probe at a high temporal resolution revealed that reaction times oscillated at a theta rhythm of 6 Hz. Moreover, oscillations showed an anti-phase relationship between memorized but uncued positions on the same versus other object as the cued position, suggesting that attentional prioritization fluctuated rhythmically in an object-based manner. Our results demonstrate the highly rhythmic nature of attentional selection in working memory. Moreover, the striking similarity between rhythmic attentional selection of mental representations and perceptual information suggests that attentional oscillations are a general mechanism of information processing in human cognition. These findings have important implications for current, attention-based models of working memory.
Can prediction error explain predictability effects on the N1 during picture-word verification?
(2023)
Do early effects of predictability in visual word recognition reflect prediction error? Electrophysiological research investigating word processing has demonstrated predictability effects in the N1, or first negative component of the event-related potential (ERP). However, findings regarding the magnitude of effects and potential interactions of predictability with lexical variables have been inconsistent. Moreover, past studies have typically used categorical designs with relatively small samples and relied on by-participant analyses. Nevertheless, reports have generally shown that predicted words elicit less negative-going (i.e., lower amplitude) N1s, a pattern consistent with a simple predictive coding account. In our preregistered study, we tested this account via the interaction between prediction magnitude and certainty. A picture-word verification paradigm was implemented in which pictures were followed by tightly matched picture-congruent or picture-incongruent written nouns. The predictability of target (picture-congruent) nouns was manipulated continuously based on norms of association between a picture and its name. ERPs from 68 participants revealed a pattern of effects opposite to that expected under a simple predictive coding framework.
Selective attention implements preferential routing of attended stimuli, likely through increasing the influence of the respective synaptic inputs on higher-area neurons. As the inputs of competing stimuli converge onto postsynaptic neurons, presynaptic circuits might offer the best target for attentional top-down influences. If those influences enabled presynaptic circuits to selectively entrain postsynaptic neurons, this might explain selective routing. Indeed, when two visual stimuli induce two gamma rhythms in V1, only the gamma induced by the attended stimulus entrains gamma in V4. Here, we modeled induced responses with a Dynamic Causal Model for Cross-Spectral Densities and found that selective entrainment can be explained by attentional modulation of intrinsic V1 connections. Specifically, local inhibition was decreased in the granular input layer and increased in the supragranular output layer of the V1 circuit that processed the attended stimulus. Thus, presynaptic attentional influences and ensuing entrainment were sufficient to mediate selective routing.
Selective attention implements preferential routing of attended stimuli, likely through increasing the influence of the respective synaptic inputs on higher-area neurons. As the inputs of competing stimuli converge onto postsynaptic neurons, presynaptic circuits might offer the best target for attentional top-down influences. If those influences enabled presynaptic circuits to selectively entrain postsynaptic neurons, this might lead to selective routing. Indeed, when two visual stimuli induce two gamma rhythms in V1, only the gamma induced by the attended stimulus entrains gamma in V4. Here, we modeled this selective entrainment with a Dynamic Causal Model for Cross-Spectral Densities and found that it can be explained by attentional modulation of intrinsic V1 connections. Specifically, local inhibition was decreased in the granular input layer and increased in the supragranular output layer of the V1 circuit that processed the attended stimulus. Thus, presynaptic attentional influences and ensuing entrainment were sufficient to mediate selective routing.
Successful consolidation of associative memories relies on the coordinated interplay of slow oscillations and sleep spindles during non-rapid eye movement (NREM) sleep, enabling the transfer of labile information from the hippocampus to permanent memory stores in the neocortex. During senescence, the decline of the structural and functional integrity of the hippocampus and neocortical regions is paralleled by changes of the physiological events that stabilize and enhance associative memories during NREM sleep. However, the currently available evidence is inconclusive if and under which circumstances aging impacts memory consolidation. By tracing the encoding quality of single memories in individual participants, we demonstrate that previous learning determines the extent of age-related impairments in memory consolidation. Specifically, the detrimental effects of aging on memory maintenance were greatest for mnemonic contents of medium encoding quality, whereas memory gain of weakly encoded memories did not differ by age. Using multivariate techniques, we identified profiles of alterations in sleep physiology and brain structure characteristic for increasing age. Importantly, while both ‘aged’ sleep and ‘aged’ brain structure profiles were associated with reduced memory maintenance, inter-individual differences in neither sleep nor structural brain integrity qualified as the driving force behind age differences in sleep-dependent consolidation in the present study.
Precise slow oscillation-spindle coupling promotes memory consolidation in younger and older adults
(2018)
Memory consolidation during sleep relies on the precisely timed interaction of rhythmic neural events. Here, we investigate differences in slow oscillations (SO) and sleep spindles (SP) and their coupling across the adult human lifespan and ask whether observed alterations relate to the ability to retain associative memories across sleep. We demonstrate that the fine-tuned SO–SP coupling that is present in younger adults diffuses with advanced age and shifts both in time and frequency. Crucially, we show that the tight precision of SO–SP coupling promotes memory consolidation in younger and older adults, and that brain integrity in source regions for the generation of SOs and SPs reinforces this beneficial SO–SP coupling in old age. Our results reveal age-related differences in SO–SP coupling in healthy elderly individuals. Furthermore, they broaden our understanding of the conditions and the functional significance of SO–SP coupling across the entire adult lifespan.
Many cross-sectional findings suggest that volumes of specific hippocampal subfields increase in middle childhood and early adolescence. In contrast, a small number of available longitudinal studies observed decreased volumes in most subfields over this age range. Further, it remains unknown whether structural changes in development are associated with corresponding gains in children’s memory. Here we report cross-sectional age differences in children’s hippocampal subfield volumes together with longitudinal developmental trajectories and their relationships with memory performance. In two waves, 109 healthy participants aged 6 to 10 years (wave 1: MAge=7.25, wave 2: MAge=9.27) underwent high-resolution magnetic resonance imaging to assess hippocampal subfield volumes, and completed cognitive tasks assessing hippocampus dependent memory processes. We found that cross-sectional age-associations and longitudinal developmental trends in hippocampal subfield volumes were highly discrepant, both by subfields and in direction. Further, volumetric changes were largely unrelated to changes in memory, with the exception that increase in subiculum volume was associated with gains in spatial memory. Importantly, the observed longitudinal patterns of brain-cognition coupling could not be inferred from cross-sectional findings. We discuss potential sources of these discrepancies. This study underscores that children’s structural brain development and its relationship to cognition cannot be inferred from cross-sectional age comparisons.
Highlights
The subiculum undergoes volumetric increase between 6-10 years of age
Change across two years in CA1-2 and DG-CA3 was not observed in this age window
Change across two years did not reflect age differences spanning two years
Cross-sectional and longitudinal slopes in stark contrast for hippocampal subfields
Longitudinal brain-cognition coupling cannot be inferred from cross-sectional data
Age-related memory decline is associated with changes in neural functioning but little is known about how aging affects the quality of information representation in the brain. Whereas a long-standing hypothesis of the aging literature links cognitive impairments to less distinct neural representations in old age, memory studies have shown that high similarity between activity patterns benefits memory performance for the respective stimuli. Here, we addressed this apparent conflict by investigating between-item representational similarity in 50 younger (19–27 years old) and 63 older (63–75 years old) human adults (male and female) who studied scene-word associations using a mnemonic imagery strategy while electroencephalography was recorded. We compared the similarity of spatiotemporal frequency patterns elicited during encoding of items with different subsequent memory fate. Compared to younger adults, older adults’ memory representations were more similar to each other but items that elicited the most similar activity patterns early in the encoding trial were those that were best remembered by older adults. In contrast, young adults’ memory performance benefited from decreased similarity between earlier and later periods in the encoding trials, which might reflect their better success in forming unique memorable mental images of the joint picture–word pair. Our results advance the understanding of the representational properties that give rise to memory quality as well as how these properties change in the course of aging.
We studied oscillatory mechanisms of memory formation in 48 younger and 51 older adults in an intentional associative memory task with cued recall. While older adults showed lower memory performance than young adults, we found subsequent memory effects (SME) in alpha/beta and theta frequency bands in both age groups. Using logistic mixed effect models, we investigated whether interindividual differences in structural integrity of key memory regions could account for interindividual differences in the strength of the SME. Structural integrity of inferior frontal gyrus (IFG) and hippocampus was reduced in older adults. SME in the alpha/beta band were modulated by the cortical thickness of IFG, in line with its hypothesized role for deep semantic elaboration. Importantly, this structure–function relationship did not differ by age group. However, older adults were more frequently represented among the participants with low cortical thickness and consequently weaker SME in the alpha band. Thus, our results suggest that differences in the structural integrity of the IFG contribute not only to interindividual, but also to age differences in memory formation.
Dual coding theories of knowledge suggest that meaning is represented in the brain by a double code, which comprises language-derived representations in the Anterior Temporal Lobe and sensory-derived representations in perceptual and motor regions. This approach predicts that concrete semantic features should activate both codes, whereas abstract features rely exclusively on the linguistic code. Using magnetoencephalography (MEG), we adopted a temporally resolved multiple regression approach to identify the contribution of abstract and concrete semantic predictors to the underlying brain signal. Results evidenced early involvement of anterior-temporal and inferior-frontal brain areas in both abstract and concrete semantic information encoding. At later stages, occipito-temporal regions showed greater responses to concrete compared to abstract features. The present findings shed new light on the temporal dynamics of abstract and concrete semantic representations in the brain and suggest that the concreteness of words processed first with a transmodal/linguistic code, housed in frontotemporal brain systems, and only after with an imagistic/sensorimotor code in perceptual and motor regions.
The knowledge that brain functional connectomes are both unique and reliable has enabled behaviourally relevant inferences at a subject level. However, it is unknown whether such “fingerprints” persist under altered states of consciousness. Ayahuasca is a potent serotonergic psychedelic which elicits a widespread dysregulation of functional connectivity. Used communally in religious ceremonies, its shared use may highlight relevant novel interactions between mental state and FC inherency. Using 7T fMRI, we assessed resting-state static and dynamic FCs for 21 Santo Daime members after collective ayahuasca intake in an acute, within-subject study. Here, connectome fingerprinting revealed a shared functional space, accompanied by a spatiotemporal reallocation of keypoint edges. Importantly, we show that interindividual differences in higher-order FCs motifs are relevant to experiential phenotypes, given that they can predict perceptual drug effects. Collectively, our findings offer an example as to how individualised connectivity markers can be used to trace a subject’s functional connectome across altered states of consciousness.
The traditional view on coding in the cortex is that populations of neurons primarily convey stimulus information through the spike count. However, given the speed of sensory processing, it has been hypothesized that sensory encoding may rely on the spike-timing relationships among neurons. Here, we use a recently developed method based on Optimal Transport Theory called SpikeShip to study the encoding of natural movies by high-dimensional ensembles of neurons in visual cortex. SpikeShip is a generic measure of dissimilarity between spike train patterns based on the relative spike-timing relations among all neurons and with computational complexity similar to the spike count. We compared spike-count and spike-timing codes in up to N > 8000 neurons from six visual areas during natural video presentations. Using SpikeShip, we show that temporal spiking sequences convey substantially more information about natural movies than population spike-count vectors when the neural population size is larger than about 200 neurons. Remarkably, encoding through temporal sequences did not show representational drift both within and between blocks. By contrast, population firing rates showed better coding performance when there were few active neurons. Furthermore, the population firing rate showed memory across frames and formed a continuous trajectory across time. In contrast to temporal spiking sequences, population firing rates exhibited substantial drift across repetitions and between blocks. These findings suggest that spike counts and temporal sequences constitute two different coding schemes with distinct information about natural movies.
Human language relies on hierarchically structured syntax to facilitate efficient and robust communication. The correct processing of syntactic information is essential for successful communication between speakers. As an abstract level of language, syntax has often been studied separately from the physical form of the speech signal, thus often masking the interactions that can promote better syntactic processing in the human brain. We analyzed a MEG dataset to investigate how acoustic cues, specifically prosody, interact with syntactic operations. We examined whether prosody enhances the cortical encoding of syntactic representations. We decoded left-sided dependencies directly from brain activity and evaluated possible modulations of the decoding by the presence of prosodic boundaries. Our findings demonstrate that prosodic boundary presence improves the representation of left-sided dependencies, indicating the facilitative role of prosodic cues in processing abstract linguistic features. This study gives neurobiological evidence for the boosting of syntactic processing via interaction with prosody.
The development of binocular vision is an active learning process comprising the development of disparity tuned neurons in visual cortex and the establishment of precise vergence control of the eyes. We present a computational model for the learning and self-calibration of active binocular vision based on the Active Efficient Coding framework, an extension of classic efficient coding ideas to active perception. Under normal rearing conditions, the model develops disparity tuned neurons and precise vergence control, allowing it to correctly interpret random dot stereogramms. Under altered rearing conditions modeled after neurophysiological experiments, the model qualitatively reproduces key experimental findings on changes in binocularity and disparity tuning. Furthermore, the model makes testable predictions regarding how altered rearing conditions impede the learning of precise vergence control. Finally, the model predicts a surprising new effect that impaired vergence control affects the statistics of orientation tuning in visual cortical neurons.
Mounting evidence suggests that perception depends on a largely-feedforward brain network. However, the discrepancy between (i) the latency of the corresponding feedforward responses (150-200 ms) and (ii) the time it takes human subjects to recognize brief images (often >500 ms) suggests that recurrent neuronal activity is critical to visual processing. Here, we use magneto-encephalography to localize, track and decode the feedforward and recurrent responses elicited by brief presentations of variably-ambiguous letters and digits. We first confirm that these stimuli trigger, within the first 200 ms, a feedforward response in the ventral and dorsal cortical pathways. The subsequent activity is distributed across temporal, parietal and prefrontal cortices and leads to a slow and incremental cascade of representations culminating in action-specific motor signals. We introduce an analytical framework to show that these brain responses are best accounted for by a hierarchy of recurrent neural assemblies. An accumulation of computational delays across specific processing stages explains subjects’ reaction times. Finally, the slow convergence of neural representations towards perceptual categories is quickly followed by all-or-none motor decision signals. Together, these results show how recurrent processes generate, over extended time periods, a cascade of hierarchical decisions that ultimately predicts subjects’ perceptual reports.
Several studies have probed perceptual performance at different times after a self-paced motor action and found frequency-specific modulations of perceptual performance phase-locked to the action. Such action-related modulation has been reported for various frequencies and modulation strengths. In an attempt to establish a basic effect at the population level, we had a relatively large number of participants (n=50) perform a self-paced button press followed by a detection task at threshold, and we applied both fixed- and random-effects tests. The combined data of all trials and participants surprisingly did not show any significant action-related modulation. However, based on previous studies, we explored the possibility that such modulation depends on the participant’s internal state. Indeed, when we split trials based on performance in neighboring trials, then trials in periods of low performance showed an action-related modulation at ≈17 Hz. When we split trials based on the performance in the preceding trial, we found that trials following a “miss” showed an action-related modulation at ≈17 Hz. Finally, when we split participants based on their false-alarm rate, we found that participants with no false alarms showed an action-related modulation at ≈17 Hz. All these effects were significant in random-effects tests, supporting an inference on the population. Together, these findings indicate that action-related modulations are not always detectable. However, the results suggest that specific internal states such as lower attentional engagement and/or higher decision criterion are characterized by a modulation in the beta-frequency range.
Several recent studies investigated the rhythmic nature of cognitive processes that lead to perception and behavioral report. These studies used different methods, and there has not yet been an agreement on a general standard. Here, we present a way to test and quantitatively compare these methods. We simulated behavioral data from a typical experiment and analyzed these data with several methods. We applied the main methods found in the literature, namely sine-wave fitting, the Discrete Fourier Transform (DFT) and the Least Square Spectrum (LSS). DFT and LSS can be applied both on the averaged accuracy time course and on single trials. LSS is mathematically equivalent to DFT in the case of regular, but not irregular sampling - which is more common. LSS additionally offers the possibility to take into account a weighting factor which affects the strength of the rhythm, such as arousal. Statistical inferences were done either on the investigated sample (fixed-effect) or on the population (random-effect) of simulated participants. Multiple comparisons across frequencies were corrected using False-Discovery-Rate, Bonferroni, or the Max-Based approach. To perform a quantitative comparison, we calculated Sensitivity, Specificity and D-prime of the investigated analysis methods and statistical approaches. Within the investigated parameter range, single-trial methods had higher sensitivity and D-prime than the methods based on the averaged-accuracy-time-course. This effect was further increased for a simulated rhythm of higher frequency. If an additional (observable) factor influenced detection performance, adding this factor as weight in the LSS further improved Sensitivity and D-prime. For multiple comparison correction, the Max-Based approach provided the highest Specificity and D-prime, closely followed by the Bonferroni approach. Given a fixed total amount of trials, the random-effect approach had higher D-prime when trials were distributed over a larger number of participants, even though this gave less trials per participant. Finally, we present the idea of using a dampened sinusoidal oscillator instead of a simple sinusoidal function, to further improve the fit to behavioral rhythmicity observed after a reset event.
Analyzing non-invasive recordings of electroencephalography (EEG) and magnetoencephalography (MEG) directly in sensor space, using the signal from individual sensors, is a convenient and standard way of working with this type of data. However, volume conduction introduces considerable challenges for sensor space analysis. While the general idea of signal mixing due to volume conduction in EEG/MEG is recognized, the implications have not yet been clearly exemplified. Here, we illustrate how different types of activity overlap on the level of individual sensors. We show spatial mixing in the context of alpha rhythms, which are known to have generators in different areas of the brain. Using simulations with a realistic 3D head model and lead field and data analysis of a large resting-state EEG dataset, we show that electrode signals can be differentially affected by spatial mixing by computing a sensor complexity measure. While prominent occipital alpha rhythms result in less heterogeneous spatial mixing on posterior electrodes, central electrodes show a diversity of rhythms present. This makes the individual contributions, such as the sensorimotor mu-rhythm and temporal alpha rhythms, hard to disentangle from the dominant occipital alpha. Additionally, we show how strong occipital rhythms rhythms can contribute the majority of activity to frontal channels, potentially compromising analyses that are solely conducted in sensor space. We also outline specific consequences of signal mixing for frequently used assessment of power, power ratios and connectivity profiles in basic research and for neurofeedback application. With this work, we hope to illustrate the effects of volume conduction in a concrete way, such that the provided practical illustrations may be of use to EEG researchers to in order to evaluate whether sensor space is an appropriate choice for their topic of investigation.
The human brain achieves visual object recognition through multiple stages of nonlinear transformations operating at a millisecond scale. To predict and explain these rapid transformations, computational neuroscientists employ machine learning modeling techniques. However, state-of-the-art models require massive amounts of data to properly train, and to the present day there is a lack of vast brain datasets which extensively sample the temporal dynamics of visual object recognition. Here we collected a large and rich dataset of high temporal resolution EEG responses to images of objects on a natural background. This dataset includes 10 participants, each with 82,160 trials spanning 16,740 image conditions. Through computational modeling we established the quality of this dataset in five ways. First, we trained linearizing encoding models that successfully synthesized the EEG responses to arbitrary images. Second, we correctly identified the recorded EEG data image conditions in a zero-shot fashion, using EEG synthesized responses to hundreds of thousands of candidate image conditions. Third, we show that both the high number of conditions as well as the trial repetitions of the EEG dataset contribute to the trained models’ prediction accuracy. Fourth, we built encoding models whose predictions well generalize to novel participants. Fifth, we demonstrate full end-to-end training of randomly initialized DNNs that output M/EEG responses for arbitrary input images. We release this dataset as a tool to foster research in visual neuroscience and computer vision.
BOLD signatures of sleep
(2019)
Sleep can be distinguished from wake by changes in brain electrical activity, typically assessed using electroencephalography (EEG). The hallmark of non-rapid-eye-movement sleep are two major EEG events: slow waves and spindles. Here we sought to identify possible signatures of sleep in brain hemodynamic activity, using simultaneous fMRI-EEG. We found that, during the transition from wake to sleep, blood-oxygen-level-dependent (BOLD) activity evolved from a mixed-frequency pattern to one dominated by two distinct oscillations: a low-frequency (~0.05Hz) oscillation prominent in light sleep and a high-frequency (~0.17Hz) oscillation in deep sleep. The two BOLD oscillations correlated with the occurrences of spindles and slow waves, respectively. They were detectable across the whole brain, cortically and subcortically, but had different regional distributions and opposite onset patterns. These spontaneous BOLD oscillations provide fMRI signatures of basic sleep processes, which may be employed to study human sleep at spatial resolution and brain coverage not achievable using EEG.
Interest in time-resolved connectivity in fMRI has grown rapidly in recent years. The most widely used technique for studying connectivity changes over time utilizes a sliding windows approach. There has been some debate about the utility of shorter versus longer windows, the use of fixed versus adaptive windows, as well as whether observed resting state dynamics during wakefulness may be predominantly due to changes in sleep state and subject head motion. In this work we use an independent component analysis (ICA)-based pipeline applied to concurrent EEG/fMRI data collected during wakefulness and various sleep stages and show: 1) connectivity states obtained from clustering sliding windowed correlations of resting state functional network time courses well classify the sleep states obtained from EEG data, 2) using shorter sliding windows instead of longer non-overlapping windows improves the ability to capture transition dynamics even at windows as short as 30 seconds, 3) motion appears to be mostly associated with one of the states rather than spread across all of them 4) a fixed tapered sliding window approach outperforms an adaptive dynamic conditional correlation approach, and 5) consistent with prior EEG/fMRI work, we identify evidence of multiple states within the wakeful condition which are able to be classified with high accuracy. Classification of wakeful only states suggest the presence of time-varying changes in connectivity in fMRI data beyond sleep state or motion. Results also inform about advantageous technical choices, and the identification of different clusters within wakefulness that are separable suggest further studies in this direction.
Brookshire (2022) claims that previous analyses of periodicity in detection performance after a reset event suffer from extreme false-positive rates. Here we show that this conclusion is based on an incorrect implemention of a null-hypothesis of aperiodicity, and that a correct implementation confirms low false-positive rates. Furthermore, we clarify that the previously used method of shuffling-in-time, and thereby shuffling-in-phase, cleanly implements the null hypothesis of no temporal structure after the reset, and thereby of no phase locking to the reset. Moving from a corresponding phase-locking spectrum to an inference on the periodicity of the underlying process can be accomplished by parameterizing the spectrum. This can separate periodic from non-periodic components, and quantify the strength of periodicity.
Cognition requires the dynamic modulation of effective connectivity, i.e. the modulation of the postsynaptic neuronal response to a given input. If postsynaptic neurons are rhythmically active, this might entail rhythmic gain modulation, such that inputs synchronized to phases of high gain benefit from enhanced effective connectivity. We show that visually induced gamma-band activity in awake macaque area V4 rhythmically modulates responses to unpredictable stimulus events. This modulation exceeded a simple additive superposition of a constant response onto ongoing gamma-rhythmic firing, demonstrating the modulation of multiplicative gain. Gamma phases leading to strongest neuronal responses also led to shortest behavioral reaction times, suggesting functional relevance of the effect. Furthermore, we find that constant optogenetic stimulation of anesthetized cat area 21a produces gamma-band activity entailing a similar gain modulation. As the gamma rhythm in area 21a did not spread backwards to area 17, this suggests that postsynaptic gamma is sufficient for gain modulation.
Synchronization has been implicated in neuronal communication, but causal evidence remains indirect. We used optogenetics to generate depolarizing currents in pyramidal neurons of cat visual cortex, emulating excitatory synaptic inputs under precise temporal control, while measuring spike output. Cortex transformed constant excitation into strong gamma-band synchronization, revealing the well-known cortical resonance. Increasing excitation with ramps increased the strength and frequency of synchronization. Slow, symmetric excitation profiles revealed hysteresis of power and frequency. Crucially, white-noise input sequences enabled causal analysis of network transmission, establishing that cortical resonance selectively transmits coherent input components. Models composed of recurrently coupled excitatory and inhibitory units uncovered a crucial role of feedback inhibition and suggest that hysteresis can arise through spike-frequency adaptation. The presented approach provides a powerful means to investigate the resonance properties of local circuits and probe how these properties transform input and shape transmission.
The gamma rhythm has been implicated in neuronal communication, but causal evidence remains indirect. We measured spike output of local neuronal networks and emulated their synaptic input through optogenetics. Opsins provide currents through somato-dendritic membranes, similar to synapses, yet under experimental control with high temporal precision. We expressed Channelrhodopsin-2 in excitatory neurons of cat visual cortex and recorded neuronal responses to light with different temporal characteristics. Sine waves of different frequencies entrained neuronal responses with a reliability that peaked for input frequencies in the gamma band. Crucially, we also presented white-noise sequences, because their temporal unpredictability enables analysis of causality. Neuronal spike output was caused specifically by the input’s gamma component. This gamma-specific transfer function is likely an emergent property of in-vivo networks with feedback inhibition. The method described here could reveal the transfer function between the input to any one and the output of any other neuronal group.
Signal transfer of visual stimuli to V4 occurs in gamma-rhythmic, pulsed information packages
(2020)
Summary Selective visual attention allows the brain to focus on behaviorally relevant information while ignoring irrelevant signals. As a possible mechanism, routing by synchronization was proposed: neural populations sending attended signals align their gamma-rhythmic activities with receiving populations, such that spikes from the senders arrive at excitability peaks of the receivers, enhancing signal transfer. Conversely, the non-attended signals arrive unaligned to the receiver’s oscillation, reducing signal transfer. Therefore, visual signals should be transferred through periodically pulsed information packages, resulting in a modulation of the stimulus content within the receiver’s activity by its gamma phase and amplitude. To test this prediction, we quantified gamma phase-specific stimulus content within neural activity from area V4 of macaques performing a visual attention task. For the attended stimulus we find enhanced stimulus content reaching its maximum near excitability peaks, with effect magnitude increasing with oscillation amplitude, establishing a functional link between selective processing and gamma activity.
Afterimages result from a prolonged exposure to still visual stimuli. They are best detectable when viewed against uniform backgrounds and can persist for multiple seconds. Consequently, the dynamics of afterimages appears to be slow by their very nature. To the contrary, we report here that about 50% of an afterimage intensity can be erased rapidly—within less than a second. The prerequisite is that subjects view a rich visual content to erase the afterimage; fast erasure of afterimages does not occur if subjects view a blank screen. Moreover, we find evidence that fast removal of afterimages is a skill learned with practice as our subjects were always more effective in cleaning up afterimages in later parts of the experiment. These results can be explained by a tri-level hierarchy of adaptive mechanisms, as has been proposed by the theory of practopoiesis.
Learning in the eyes: specific changes in gaze patterns track explicit and implicit visual learning
(2020)
What is the link between eye movements and sensory learning? Although some theories have argued for a permanent and automatic interaction between what we know and where we look, which continuously modulates human information- gathering behavior during both implicit and explicit learning, there exist surprisingly little evidence supporting such an ongoing interaction. We used a pure form of implicit learning called visual statistical learning and manipulated the explicitness of the task to explore how learning and eye movements interact. During both implicit exploration and explicit visual learning of unknown composite visual scenes, eye movement patterns systematically changed in accordance with the underlying statistical structure of the scenes. Moreover, the degree of change was directly correlated with the amount of knowledge the observers acquired. Our results provide the first evidence for an ongoing and specific interaction between hitherto accumulated knowledge and eye movements during both implicit and explicit learning.
Electroencephalography (EEG) has been used for decades to identify neurocognitive processes related to intelligence. Evidence is accumulating for associations with neural markers of higher-order cognitive processes (e.g., working memory); however, whether associations are specific to complex processes or also relate to earlier processing stages remains unclear. Addressing these issues has implications for improving our understanding of intelligence and its neural correlates. The mismatch negativity (MMN) is an event-related brain potential (ERP) that is elicited when, within a series of frequent standard stimuli, rare deviant stimuli are presented. As stimuli are typically presented outside the focus of attention, the MMN is suggested to capture automatic pre-attentive discrimination processes. However, the MMN and its relation to intelligence has largely only been studied in the auditory domain, thus preventing conclusions about the involvement of automatic discrimination processes in humans’ dominant sensory modality vision. Electroencephalography was recorded from 50 healthy participants during a passive visual oddball task that presented simple sequence violations as well as deviations within a more complex hidden pattern. Signed area amplitudes and fractional area latencies of the visual mismatch negativity (vMMN) were calculated with and without Laplacian transformation. Correlations between vMMN and intelligence (Raven’s Advanced Progressive Matrices) were of negligible to small effect sizes, differed critically between measurement approaches, and Bayes Factors provided anecdotal to substantial evidence for the absence of an association. We discuss differences between the auditory and visual MMN, the implications of different measurement approaches, and offer recommendations for further research in this evolving field.
How much data do we need? Lower bounds of brain activation states to predict human cognitive ability
(2022)
Human functional brain connectivity can be temporally decomposed into states of high and low cofluctuation, defined as coactivation of brain regions over time. Despite their low frequency of occurrence, states of particularly high cofluctuation have been shown to reflect fundamentals of intrinsic functional network architecture (derived from resting-state fMRI) and to be highly subject-specific. However, it is currently unclear whether such network-defining states of high cofluctuation also contribute to individual variations in cognitive abilities – which strongly rely on the interactions among distributed brain regions. By introducing CMEP, an eigenvector-based prediction framework, we show that functional connectivity estimates from as few as 20 temporally separated time frames (< 3% of a 10 min resting-state fMRI scan) are significantly predictive of individual differences in intelligence (N = 281, p < .001). In contrast and against previous expectations, individual’s network-defining time frames of particularly high cofluctuation do not achieve significant prediction of intelligence. Multiple functional brain networks contribute to the prediction, and all results replicate in an independent sample (N = 831). Our results suggest that although fundamentals of person-specific functional connectomes can be derived from few time frames of highest brain connectivity, temporally distributed information is necessary to extract information about cognitive abilities from functional connectivity time series. This information, however, is not restricted to specific connectivity states, like network-defining high-cofluctuation states, but rather reflected across the entire length of the brain connectivity time series.
Probing the association between resting state brain network dynamics and psychological resilience
(2021)
Abstract
This study aimed at replicating a previously reported negative correlation between node flexibility and psychological resilience, i.e., the ability to retain mental health in the face of stress and adversity. To this end, we used multiband resting-state BOLD fMRI (TR = .675 sec) from 52 participants who had filled out three psychological questionnaires assessing resilience. Time-resolved functional connectivity was calculated by performing a sliding window approach on averaged time series parcellated according to different established atlases. Multilayer modularity detection was performed to track network reconfigurations over time and node flexibility was calculated as the number of times a node changes community assignment. In addition, node promiscuity (the fraction of communities a node participates in) and node degree (as proxy for time-varying connectivity) were calculated to extend previous work. We found no substantial correlations between resilience and node flexibility. We observed a small number of correlations between the two other brain measures and resilience scores, that were however very inconsistently distributed across brain measures, differences in temporal sampling, and parcellation schemes. This heterogeneity calls into question the existence of previously postulated associations between resilience and brain network flexibility and highlights how results may be influenced by specific analysis choices.
Author Summary We tested the replicability and generalizability of a previously proposed negative association between dynamic brain network reconfigurations derived from multilayer modularity detection (node flexibility) and psychological resilience. Using multiband resting-state BOLD fMRI data and exploring several parcellation schemes, sliding window approaches, and temporal resolutions of the data, we could not replicate previously reported findings regarding the association between node flexibility and resilience. By extending this work to other measures of brain dynamics (node promiscuity, degree) we observe a rather inconsistent pattern of correlations with resilience, that strongly varies across analysis choices. We conclude that further research is needed to understand the network neuroscience basis of mental health and discuss several reasons that may account for the variability in results.
Word familiarity and predictive context facilitate visual word processing, leading to faster recognition times and reduced neuronal responses. Previously, models with and without top-down connections, including lexical-semantic, pre-lexical (e.g., orthographic/ phonological), and visual processing levels were successful in accounting for these facilitation effects. Here we systematically assessed context-based facilitation with a repetition priming task and explicitly dissociated pre-lexical and lexical processing levels using a pseudoword familiarization procedure. Experiment 1 investigated the temporal dynamics of neuronal facilitation effects with magnetoencephalography (MEG; N=38 human participants) while Experiment 2 assessed behavioral facilitation effects (N=24 human participants). Across all stimulus conditions, MEG demonstrated context-based facilitation across multiple time windows starting at 100 ms, in occipital brain areas. This finding indicates context based-facilitation at an early visual processing level. In both experiments, we furthermore found an interaction of context and lexical familiarity, such that stimuli with associated meaning showed the strongest context-dependent facilitation in brain activation and behavior. Using MEG, this facilitation effect could be localized to the left anterior temporal lobe at around 400 ms, indicating within-level (i.e., exclusively lexical-semantic) facilitation but no top-down effects on earlier processing stages. Increased pre-lexical familiarity (in pseudowords familiarized utilizing training) did not enhance or reduce context effects significantly. We conclude that context based-facilitation is achieved within visual and lexical processing levels. Finally, by testing alternative hypotheses derived from mechanistic accounts of repetition suppression, we suggest that the facilitatory context effects found here are implemented using a predictive coding mechanism.
To characterize the left-ventral occipito-temporal cortex (lvOT) role during reading in a quantitatively explicit and testable manner, we propose the lexical categorization model (LCM). The LCM assumes that lvOT optimizes linguistic processing by allowing fast meaning access when words are familiar and filter out orthographic strings without meaning. The LCM successfully simulates benchmark results from functional brain imaging. Empirically, using functional magnetic resonance imaging, we demonstrate that quantitative LCM simulations predict lvOT activation across three studies better than alternative models. Besides, we found that word-likeness, which is assumed as input to LCM, is represented posterior to lvOT. In contrast, a dichotomous word/non-word contrast, which is assumed as the LCM’s output, could be localized to upstream frontal brain regions. Finally, we found that training lexical categorization results in more efficient reading. Thus, we propose a ventral-visual-stream processing framework for reading involving word-likeness extraction followed by lexical categorization, before meaning extraction.
To a crucial extent, the efficiency of reading results from the fact that visual word recognition is faster in predictive contexts. Predictive coding models suggest that this facilitation results from pre-activation of predictable stimulus features across multiple representational levels before stimulus onset. Still, it is not sufficiently understood which aspects of the rich set of linguistic representations that are activated during reading – visual, orthographic, phonological, and/or lexical-semantic – contribute to context-dependent facilitation. To investigate in detail which linguistic representations are pre-activated in a predictive context and how they affect subsequent stimulus processing, we combined a well-controlled repetition priming paradigm, including words and pseudowords (i.e., pronounceable nonwords), with behavioral and magnetoencephalography measurements. For statistical analysis, we used linear mixed modeling, which we found had a higher statistical power compared to conventional multivariate pattern decoding analysis. Behavioral data from 49 participants indicate that word predictability (i.e., context present vs. absent) facilitated orthographic and lexical-semantic, but not visual or phonological processes. Magnetoencephalography data from 38 participants show sustained activation of orthographic and lexical-semantic representations in the interval before processing the predicted stimulus, suggesting selective pre-activation at multiple levels of linguistic representation as proposed by predictive coding. However, we found more robust lexical-semantic representations when processing predictable in contrast to unpredictable letter strings, and pre-activation effects mainly resembled brain responses elicited when processing the expected letter string. This finding suggests that pre-activation did not result in ‘explaining away’ predictable stimulus features, but rather in a ‘sharpening’ of brain responses involved in word processing.
Most current models assume that the perceptual and cognitive processes of visual word recognition and reading operate upon neuronally coded domain-general low-level visual representations – typically oriented line representations. We here demonstrate, consistent with neurophysiological theories of Bayesian-like predictive neural computations, that prior visual knowledge of words may be utilized to ‘explain away’ redundant and highly expected parts of the visual percept. Subsequent processing stages, accordingly, operate upon an optimized representation of the visual input, the orthographic prediction error, highlighting only the visual information relevant for word identification. We show that this optimized representation is related to orthographic word characteristics, accounts for word recognition behavior, and is processed early in the visual processing stream, i.e., in V4 and before 200 ms after word-onset. Based on these findings, we propose that prior visual-orthographic knowledge is used to optimize the representation of visually presented words, which in turn allows for highly efficient reading processes.
How is semantic information stored in the human mind and brain? Some philosophers and cognitive scientists argue for vectorial representations of concepts, where the meaning of a word is represented as its position in a high-dimensional neural state space. At the intersection of natural language processing and artificial intelligence, a class of very successful distributional word vector models has developed that can account for classic EEG findings of language, i.e., the ease vs. difficulty of integrating a word with its sentence context. However, models of semantics have to account not only for context-based word processing, but should also describe how word meaning is represented. Here, we investigate whether distributional vector representations of word meaning can model brain activity induced by words presented without context. Using EEG activity (event-related brain potentials) collected while participants in two experiments (English, German) read isolated words, we encode and decode word vectors taken from the family of prediction-based word2vec algorithms. We find that, first, the position of a word in vector space allows the prediction of the pattern of corresponding neural activity over time, in particular during a time window of 300 to 500 ms after word onset. Second, distributional models perform better than a human-created taxonomic baseline model (WordNet), and this holds for several distinct vector-based models. Third, multiple latent semantic dimensions of word meaning can be decoded from brain activity. Combined, these results suggest that empiricist, prediction-based vectorial representations of meaning are a viable candidate for the representational architecture of human semantic knowledge.
The outstanding speed of language comprehension necessitates a highly efficient implementation of cognitive-linguistic processes. The domain-general theory of Predictive Coding suggests that our brain solves this problem by continuously forming linguistic predictions about expected upcoming input. The neurophysiological implementation of these predictive linguistic processes, however, is not yet understood. Here, we use EEG (human participants, both sexes) to investigate the existence and nature of online-generated, category-level semantic representations during sentence processing. We conducted two experiments in which some nouns – embedded in a predictive spoken sentence context – were unexpectedly delayed by 1 second. Target nouns were either abstract/concrete (Experiment 1) or animate/inanimate (Experiment 2). We hypothesized that if neural prediction error signals following (temporary) omissions carry specific information about the stimulus, the semantic category of the upcoming target word is encoded in brain activity prior to its presentation. Using time-generalized multivariate pattern analysis, we demonstrate significant decoding of word category from silent periods directly preceding the target word, in both experiments. This provides direct evidence for predictive coding during sentence processing, i.e., that information about a word can be encoded in brain activity before it is perceived. While the same semantic contrast could also be decoded from EEG activity elicited by isolated words (Experiment 1), the identified neural patterns did not generalize to pre-stimulus delay period activity in sentences. Our results not only indicate that the brain processes language predictively, but also demonstrate the nature and sentence-specificity of category-level semantic predictions preactivated during sentence comprehension.
The ability to extract regularities from the environment is arguably an adaptive characteristic of intelligent systems. In the context of speech, statistical learning is thought to be an important mechanism for language acquisition. By considering individual differences in speech auditory-motor synchronization, an independent component analysis of fMRI data revealed that the neural substrates of statistical word form learning are not fully shared across individuals. While a network of auditory and superior pre/motor regions is universally activated in the process of learning, a fronto-parietal network is instead additionally and selectively engaged by some individuals, boosting their performance. Furthermore, interfering with the use of this network via articulatory suppression (producing irrelevant speech during learning) normalizes performance across the entire sample. Our work provides novel insights on language-related statistical learning and reconciles previous contrasting findings, while highlighting the need to factor in fundamental individual differences for a precise characterization of cognitive phenomena.
Across languages, the speech signal is characterized by a predominant modulation of the amplitude spectrum between about 4.3-5.5Hz, reflecting the production and processing of linguistic information chunks (syllables, words) every ∼200ms. Interestingly, ∼200ms is also the typical duration of eye fixations during reading. Prompted by this observation, we demonstrate that German readers sample written text at ∼5Hz. A subsequent meta-analysis with 142 studies from 14 languages replicates this result, but also shows that sampling frequencies vary across languages between 3.9Hz and 5.2Hz, and that this variation systematically depends on the complexity of the writing systems (character-based vs. alphabetic systems, orthographic transparency). Finally, we demonstrate empirically a positive correlation between speech spectrum and eye-movement sampling in low-skilled readers. Based on this convergent evidence, we propose that during reading, our brain’s linguistic processing systems imprint a preferred processing rate, i.e., the rate of spoken language production and perception, onto the oculomotor system.
Precisely estimating event timing is essential for survival, yet temporal distortions are ubiquitous in our daily sensory experience. Here, we tested whether the relative position, relative duration and relative distance in time of two sequentially-organized events —standard S, with constant duration, and comparison C, varying trial-by-trial— are causal factors in generating temporal distortions. We found that temporal distortions emerge when the first event is shorter than the second event. Importantly, a significant interaction suggests that a longer ISI helps counteracting such serial distortion effect only the constant S is in first position, but not if the unpredictable C is in first position. These results suggest the existence of a perceptual bias in perceiving ordered event durations, mechanistically contributing to distortion in time perception. We simulated our behavioral results with a Bayesian model and replicated the finding that participants disproportionately expand first-position dynamic (unpredictable) short events. Our results clarify the mechanics generating time distortions by identifying a hitherto unknown duration-dependent encoding inefficiency in human serial temporal perception, akin to a strong prior that can be overridden for highly predictable sensory events but unfolds for unpredictable ones.
Research points to neurofunctional differences underlying fluent speech production in stutterers and non-stutterers. There has been considerably less work focusing on the processes that underlie stuttered speech, primarily due to the difficulty of reliably eliciting stuttering in the unnatural contexts associated with neuroimaging experiments. We used magnetoencephalography (MEG) to test the hypothesis that stuttering events result from global motor inhibition–a “freeze” response typically characterized by increased beta power in nodes of the action-stopping network. We leveraged a novel clinical interview to develop participant-specific stimuli in order to elicit a comparable amount of stuttered and fluent trials. Twenty-nine adult stutterers participated. The paradigm included a cue prior to a go signal, which allowed us to isolate processes associated with stuttered and fluent trials prior to speech initiation. During this pre-speech time window, stuttered trials were associated with greater beta power in the right pre-supplementary motor area, a key node in the action-stopping network, compared to fluent trials. Beta power in the right pre-supplementary area was related to a clinical measure of stuttering severity. We also found that anticipated words identified independently by participants were stuttered more often than those generated by the researchers, which were based on the participants’ reported anticipated sounds. This suggests that global motor inhibition results from stuttering anticipation. This study represents the largest comparison of stuttered and fluent speech to date. The findings provide a foundation for clinical trials that test the efficacy of neuromodulation on stuttering. Moreover, our study demonstrates the feasibility of using our approach for eliciting stuttering during MEG and functional magnetic resonance imaging experiments so that the neurobiological bases of stuttered speech can be further elucidated.
When speech is too fast, the tracking of the acoustic signal along the auditory pathway deteriorates, leading to suboptimal speech segmentation and decoding of speech information. Thus, speech comprehension is limited by the temporal constraints of the auditory system. Here we ask whether individual differences in auditory-motor coupling strength in part shape these temporal constraints. In two behavioral experiments, we characterize individual differences in the comprehension of naturalistic speech as function of the individual synchronization between the auditory and motor systems and the preferred frequencies of the systems. Obviously, speech comprehension declined at higher speech rates. Importantly, however, both higher auditory-motor synchronization and higher spontaneous speech motor production rates were predictive of better speech-comprehension performance. Furthermore, performance increased with higher working memory capacity (Digit Span) and higher linguistic, model-based sentence predictability – particularly so at higher speech rates and for individuals with high auditory-motor synchronization. These findings support the notion of an individual preferred auditory– motor regime that allows for optimal speech processing. The data provide evidence for a model that assigns a central role to motor-system-dependent individual flexibility in continuous speech comprehension.
Speech imagery (the ability to generate internally quasi-perceptual experiences of speech) is a fundamental ability linked to cognitive functions such as inner speech, phonological working memory, and predictive processing. Speech imagery is also considered an ideal tool to test theories of overt speech. The study of speech imagery is challenging, primarily because of the absence of overt behavioral output as well as the difficulty in temporally aligning imagery events across trials and individuals. We used magnetoencephalography (MEG) paired with temporal-generalization-based neural decoding and a simple behavioral protocol to determine the processing stages underlying speech imagery. We monitored participants’ lip and jaw micromovements during mental imagery of syllable production using electromyography. Decoding participants’ imagined syllables revealed a sequence of task-elicited representations. Importantly, participants’ micromovements did not discriminate between syllables. The decoded sequence of neuronal patterns maps well onto the predictions of current computational models of overt speech motor control and provides evidence for hypothesized internal and external feedback loops for speech planning and production, respectively. Additionally, the results expose the compressed nature of representations during planning which contrasts with the natural rate at which internal productions unfold. We conjecture that the same sequence underlies the motor-based generation of sensory predictions that modulate speech perception as well as the hypothesized articulatory loop of phonological working memory. The results underscore the potential of speech imagery, based on new experimental approaches and analytical methods, and further pave the way for successful non-invasive brain-computer interfaces.
Music, like language, is characterized by hierarchically organized structure that unfolds over time. Music listening therefore requires not only the tracking of notes and beats but also internally constructing high-level musical structures or phrases and anticipating incoming contents. Unlike for language, mechanistic evidence for online musical segmentation and prediction at a structural level is sparse. We recorded neurophysiological data from participants listening to music in its original forms as well as in manipulated versions with locally or globally reversed harmonic structures. We discovered a low-frequency neural component that modulated the neural rhythms of beat tracking and reliably parsed musical phrases. We next identified phrasal phase precession, suggesting that listeners established structural predictions from ongoing listening experience to track phrasal boundaries. The data point to brain mechanisms that listeners use to segment continuous music at the phrasal level and to predict abstract structural features of music.
Mental imagery provides an essential simulation tool for remembering the past and planning the future, with its strength affecting both cognition and mental health. Research suggests that neural activity spanning prefrontal, parietal, temporal, and visual areas supports the generation of mental images. Exactly how this network controls the strength of visual imagery remains unknown. Here, brain imaging and transcranial magnetic phosphene data show that lower resting activity and excitability levels in early visual cortex (V1-V3) predict stronger sensory imagery. Electrically decreasing visual cortex excitability using tDCS increases imagery strength, demonstrating a causative role of visual cortex excitability in controlling visual imagery. These data suggest a neurophysiological mechanism of cortical excitability involved in controlling the strength of mental images.
Spike count correlations (SCCs) are ubiquitous in sensory cortices, are characterized by rich structure and arise from structured internal interactions. Yet, most theories of visual perception focus exclusively on the mean responses of individual neurons. Here, we argue that feedback interactions in primary visual cortex (V1) establish the context in which individual neurons process complex stimuli and that changes in visual context give rise to stimulus-dependent SCCs. Measuring V1 population responses to natural scenes in behaving macaques, we show that the fine structure of SCCs is stimulus-specific and variations in response correlations across-stimuli are independent of variations in response means. Moreover, we demonstrate that stimulus-specificity of SCCs in V1 can be directly manipulated by controlling the high-order structure of synthetic stimuli. We propose that stimulus-specificity of SCCs is a natural consequence of hierarchical inference where inferences on the presence of high-level image features modulate inferences on the presence of low-level features.
Natural scene responses in the primary visual cortex are modulated simultaneously by attention and by contextual signals about scene statistics stored across the connectivity of the visual processing hierarchy. We hypothesize that attentional and contextual top-down signals interact in V1, in a manner that primarily benefits the representation of natural visual stimuli, rich in high-order statistical structure. Recording from two macaques engaged in a spatial attention task, we show that attention enhances the decodability of stimulus identity from population responses evoked by natural scenes but, critically, not by synthetic stimuli in which higher-order statistical regularities were eliminated. Attentional enhancement of stimulus decodability from population responses occurs in low dimensional spaces, as revealed by principal component analysis, suggesting an alignment between the attentional and the natural stimulus variance. Moreover, natural scenes produce stimulus-specific oscillatory responses in V1, whose power undergoes a global shift from low to high frequencies with attention. We argue that attention and perception share top-down pathways, which mediate hierarchical interactions optimized for natural vision.
Grasping the meaning of everyday visual events is a fundamental feat of human intelligence that hinges on diverse neural processes ranging from vision to higher-level cognition. Deciphering the neural basis of visual event understanding requires rich, extensive, and appropriately designed experimental data. However, this type of data is hitherto missing. To fill this gap, we introduce the BOLD Moments Dataset (BMD), a large dataset of whole-brain fMRI responses to over 1,000 short (3s) naturalistic video clips and accompanying metadata. We show visual events interface with an array of processes, extending even to memory, and we reveal a match in hierarchical processing between brains and video-computable deep neural networks. Furthermore, we showcase that BMD successfully captures temporal dynamics of visual events at second resolution. BMD thus establishes a critical groundwork for investigations of the neural basis of visual event understanding.
Orientation hypercolumns in the visual cortex are delimited by the repeating pinwheel patterns of orientation selective neurons. We design a generative model for visual cortex maps that reproduces such orientation hypercolumns as well as ocular dominance maps while preserving retinotopy. The model uses a neural placement method based on t–distributed stochastic neighbour embedding (t–SNE) to create maps that order common features in the connectivity matrix of the circuit. We find that, in our model, hypercolumns generally appear with fixed cell numbers independently of the overall network size. These results would suggest that existing differences in absolute pinwheel densities are a consequence of variations in neuronal density. Indeed, available measurements in the visual cortex indicate that pinwheels consist of a constant number of ∼30, 000 neurons. Our model is able to reproduce a large number of characteristic properties known for visual cortex maps. We provide the corresponding software in our MAPStoolbox for Matlab.
Reducing neuronal size results in less cell membrane and therefore lower input conductance. Smaller neurons are thus more excitable as seen in their voltage responses to current injections in the soma. However, the impact of a neuron’s size and shape on its voltage responses to synaptic activation in dendrites is much less understood. Here we use analytical cable theory to predict voltage responses to distributed synaptic inputs and show that these are entirely independent of dendritic length. For a given synaptic density, a neuron’s response depends only on the average dendritic diameter and its intrinsic conductivity. These results remain true for the entire range of possible dendritic morphologies irrespective of any particular arborisation complexity. Also, spiking models result in morphology invariant numbers of action potentials that encode the percentage of active synapses. Interestingly, in contrast to spike rate, spike times do depend on dendrite morphology. In summary, a neuron’s excitability in response to synaptic inputs is not affected by total dendrite length. It rather provides a homeostatic input-output relation that specialised synapse distributions, local non-linearities in the dendrites and synaptic plasticity can modulate. Our work reveals a new fundamental principle of dendritic constancy that has consequences for the overall computation in neural circuits.
Excess neuronal branching allows for innervation of specific dendritic compartments in cortex
(2019)
The connectivity of cortical microcircuits is a major determinant of brain function; defining how activity propagates between different cell types is key to scaling our understanding of individual neuronal behaviour to encompass functional networks. Furthermore, the integration of synaptic currents within a dendrite depends on the spatial organisation of inputs, both excitatory and inhibitory. We identify a simple equation to estimate the number of potential anatomical contacts between neurons; finding a linear increase in potential connectivity with cable length and maximum spine length, and a decrease with overlapping volume. This enables us to predict the mean number of candidate synapses for reconstructed cells, including those realistically arranged. We identify an excess of putative connections in cortical data, with densities of neurite higher than is necessary to reliably ensure the possible implementation of any given connection. We show that potential contacts allow the particular implementation of connectivity at a subcellular level.
The brain adapts to the sensory environment. For example, simple sensory exposure can modify the response properties of early sensory neurons. How these changes affect the overall encoding and maintenance of stimulus information across neuronal populations remains unclear. We perform parallel recordings in the primary visual cortex of anesthetized cats and find that brief, repetitive exposure to structured visual stimuli enhances stimulus encoding by decreasing the selectivity and increasing the range of the neuronal responses that persist after stimulus presentation. Low-dimensional projection methods and simple classifiers demonstrate that visual exposure increases the segregation of persistent neuronal population responses into stimulus-specific clusters. These observed refinements preserve the representational details required for stimulus reconstruction and are detectable in post-exposure spontaneous activity. Assuming response facilitation and recurrent network interactions as the core mechanisms underlying stimulus persistence, we show that the exposure-driven segregation of stimulus responses can arise through strictly local plasticity mechanisms, also in the absence of firing rate changes. Our findings provide evidence for the existence of an automatic, unguided optimization process that enhances the encoding power of neuronal populations in early visual cortex, thus potentially benefiting simple readouts at higher stages of visual processing.
Abstract Trial-to-trial variability and spontaneous activity of cortical recordings have been suggested to reflect intrinsic noise. This view is currently challenged by mounting evidence for structure in these phenomena: Trial-to-trial variability decreases following stimulus onset and can be predicted by previous spontaneous activity. This spontaneous activity is similar in magnitude and structure to evoked activity and can predict decisions. Allof the observed neuronal properties described above can be accounted for, at an abstract computational level, by the sampling-hypothesis, according to which response variability reflects stimulus uncertainty. However, a mechanistic explanation at the level of neural circuit dynamics is still missing.
In this study, we demonstrate that all of these phenomena can be accounted for by a noise-free self-organizing recurrent neural network model (SORN). It combines spike-timing dependent plasticity (STDP) and homeostatic mechanisms in a deterministic network of excitatory and inhibitory McCulloch-Pitts neurons. The network self-organizes to spatio-temporally varying input sequences.
We find that the key properties of neural variability mentioned above develop in this model as the network learns to perform sampling-like inference. Importantly, the model shows high trial-to-trial variability although it is fully deterministic. This suggests that the trial-to-trial variability in neural recordings may not reflect intrinsic noise. Rather, it may reflect a deterministic approximation of sampling-like learning and inference. The simplicity of the model suggests that these correlates of the sampling theory are canonical properties of recurrent networks that learn with a combination of STDP and homeostatic plasticity mechanisms.
Author Summary Neural recordings seem very noisy. If the exact same stimulus is shown to an animal multiple times, the neural response will vary. In fact, the activity of a single neuron shows many features of a stochastic process. Furthermore, in the absence of a sensory stimulus, cortical spontaneous activity has a magnitude comparable to the activity observed during stimulus presentation. These findings have led to a widespread belief that neural activity is indeed very noisy. However, recent evidence indicates that individual neurons can operate very reliably and that the spontaneous activity in the brain is highly structured, suggesting that much of the noise may in fact be signal. One hypothesis regarding this putative signal is that it reflects a form of probabilistic inference through sampling. Here we show that the key features of neural variability can be accounted for in a completely deterministic network model through self-organization. As the network learns a model of its sensory inputs, the deterministic dynamics give rise to sampling-like inference. Our findings show that the notorious variability in neural recordings does not need to be seen as evidence for a noisy brain. Instead it may reflect sampling-like inference emerging from a self-organized learning process.
Background Corticospinal excitability depends on the current brain state. The recent development of real-time EEG-triggered transcranial magnetic stimulation (EEG-TMS) allows studying this relationship in a causal fashion. Specifically, it has been shown that corticospinal excitability is higher during the scalp surface negative EEG peak compared to the positive peak of µ-oscillations in sensorimotor cortex, as indexed by larger motor evoked potentials (MEPs) for fixed stimulation intensity.
Objective We further characterize the effect of µ-rhythm phase on the MEP input-output (IO) curve by measuring the degree of excitability modulation across a range of stimulation intensities. We furthermore seek to optimize stimulation parameters to enable discrimination of functionally relevant EEG-defined brain states.
Methods A real-time EEG-TMS system was used to trigger MEPs during instantaneous brain-states corresponding to µ-rhythm surface positive and negative peaks with five different stimulation intensities covering an individually calibrated MEP IO curve in 15 healthy participants.
Results MEP amplitude is modulated by µ-phase across a wide range of stimulation intensities, with larger MEPs at the surface negative peak. The largest relative MEP-modulation was observed for weak intensities, the largest absolute MEP-modulation for intermediate intensities. These results indicate a leftward shift of the MEP IO curve during the µ-rhythm negative peak.
Conclusion The choice of stimulation intensity influences the observed degree of corticospinal excitability modulation by µ-phase. Lower stimulation intensities enable more efficient differentiation of EEG µ-phase-defined brain states.
Active efficient coding explains the development of binocular vision and its failure in amblyopia
(2020)
The development of vision during the first months of life is an active process that comprises the learning of appropriate neural representations and the learning of accurate eye movements. While it has long been suspected that the two learning processes are coupled, there is still no widely accepted theoretical framework describing this joint development. Here we propose a computational model of the development of active binocular vision to fill this gap. The model is based on a new formulation of the Active Efficient Coding theory, which proposes that eye movements, as well as stimulus encoding, are jointly adapted to maximize the overall coding efficiency. Under healthy conditions, the model self-calibrates to perform accurate vergence and accommodation eye movements. It exploits disparity cues to deduce the direction of defocus, which leads to co-ordinated vergence and accommodation responses. In a simulated anisometropic case, where the refraction power of the two eyes differs, an amblyopia-like state develops, in which the foveal region of one eye is suppressed due to inputs from the other eye. After correcting for refractive errors, the model can only reach healthy performance levels if receptive fields are still plastic, in line with findings on a critical period for binocular vision development. Overall, our model offers a unifying conceptual framework for understanding the development of binocular vision.
Spatial attention increases both inter-areal synchronization and spike rates across the visual hierarchy. To investigate whether these attentional changes reflect distinct or common mechanisms, we performed simultaneous laminar recordings of identified cell classes in macaque V1 and V4. Enhanced V4 spike rates were expressed by both excitatory neurons and fast-spiking interneurons, and were most prominent and arose earliest in time in superficial layers, consistent with a feedback modulation. By contrast, V1-V4 gamma-synchronization reflected feedforward communication and surprisingly engaged only fast-spiking interneurons in the V4 input layer. In mouse visual cortex, we found a similar motif for optogenetically identified inhibitory-interneuron classes. Population decoding analyses further indicate that feedback-related increases in spikes rates encoded attention more reliably than feedforward-related increases in synchronization. These findings reveal distinct, cell-type-specific feedforward and feedback pathways for the attentional modulation of inter-areal synchronization and spike rates, respectively.
Developmental loss of ErbB4 in PV interneurons disrupts state-dependent cortical circuit dynamics
(2020)
GABAergic inhibition plays an important role in the establishment and maintenance of cortical circuits during development. Neuregulin 1 (Nrg1) and its interneuron-specific receptor ErbB4 are key elements of a signaling pathway critical for the maturation and proper synaptic connectivity of interneurons. Using conditional deletions of the ERBB4 gene in mice, we tested the role of this signaling pathway at two developmental timepoints in parvalbumin-expressing (PV) interneurons, the largest subpopulation of cortical GABAergic cells. Loss of ErbB4 in PV interneurons during embryonic, but not late postnatal, development leads to alterations in the activity of excitatory and inhibitory cortical neurons, along with severe disruption of cortical temporal organization. These impairments emerge by the end of the second postnatal week, prior to the complete maturation of the PV interneurons themselves. Early loss of ErbB4 in PV interneurons also results in profound dysregulation of excitatory pyramidal neuron dendritic architecture and a redistribution of spine density at the apical dendritic tuft. In association with these deficits, excitatory cortical neurons exhibit normal tuning for sensory inputs, but a loss of state-dependent modulation of the gain of sensory responses. Together these data support a key role for early developmental Nrg1/ErbB4 signaling in PV interneurons as powerful mechanism underlying the maturation of both the inhibitory and excitatory components of cortical circuits.
An important question concerning inter-areal communication in the cortex is whether these interactions are synergistic, i.e. convey information beyond what can be performed by isolated signals. Here, we dissociated cortical interactions sharing common information from those encoding complementary information during prediction error processing. To this end, we computed co-information, an information-theoretical measure that distinguishes redundant from synergistic information among brain signals. We analyzed auditory and frontal electrocorticography (ECoG) signals in three common awake marmosets and investigated to what extent event-related-potentials (ERP) and broadband (BB) dynamics exhibit redundancy and synergy for auditory prediction error signals. We observed multiple patterns of redundancy and synergy across the entire cortical hierarchy with distinct dynamics. The information conveyed by ERPs and BB signals was highly synergistic even at lower stages of the hierarchy in the auditory cortex, as well as between lower and higher areas in the frontal cortex. These results indicate that the distributed representations of prediction error signals across the cortical hierarchy can be highly synergistic.
SpikeShip: a method for fast, unsupervised discovery of high-dimensional neural spiking patterns
(2023)
Neural coding and memory formation depend on temporal spiking sequences that span high-dimensional neural ensembles. The unsupervised discovery and characterization of these spiking sequences requires a suitable dissimilarity measure to spiking patterns, which can then be used for clustering and decoding. Here, we present a new dissimilarity measure based on optimal transport theory called SpikeShip, which compares multi-neuron spiking patterns based on all the relative spike-timing relationships among neurons. SpikeShip computes the optimal transport cost to make all the relative spike timing relationships (across neurons) identical between two spiking patterns. We show that this transport cost can be decomposed into a temporal rigid translation term, which captures global latency shifts, and a vector of neuron-specific transport flows, which reflect inter-neuronal spike timing differences. SpikeShip can be effectively computed for high-dimensional neuronal ensembles, has a low (linear) computational cost that has the same order as the spike count, and is sensitive to higher-order correlations. Furthermore SpikeShip is binless, can handle any form of spike time distributions, is not affected by firing rate fluctuations, can detect patterns with a low signal-to-noise ratio, and can be effectively combined with a sliding window approach. We compare the advantages and differences between SpikeShip and other measures like SPIKE and Victor-P urpura distance. We applied SpikeShip to large-scale Neuropixel recordings during spontaneous activity and visual encoding. We show that high-dimensional spiking sequences detected via SpikeShip reliably distinguish between different natural images and different behavioral states. These spiking sequences carried complementary information to conventional firing rate codes. SpikeShip opens new avenues for studying neural coding and memory consolidation by rapid and unsupervised detection of temporal spiking patterns in high-dimensional neural ensembles.
The hippocampal formation is linked to spatial navigation, but there is little corroboration from freely-moving primates with concurrent monitoring of three-dimensional head and gaze stances. We recorded neurons and local field potentials across hippocampal regions in rhesus macaques during free foraging in an open environment while tracking their head and eye. Theta band activity was intermittently present at movement onset and modulated by saccades. Many cells were phase-locked to theta, with few showing theta phase precession. Most hippocampal neurons encoded a mixture of spatial variables beyond place fields and a negligible number showed prominent grid tuning. Spatial representations were dominated by facing location and allocentric direction, mostly in head, rather than gaze, coordinates. Importantly, eye movements strongly modulated neural activity in all regions. These findings reveal that the macaque hippocampal formation represents three-dimensional space using a multiplexed code, with head orientation and eye movement properties dominating over simple place and grid coding during free exploration.
Path integration is a sensorimotor computation that can be used to infer latent dynamical states by integrating self-motion cues. We studied the influence of sensory observation (visual/vestibular) and latent control dynamics (velocity/acceleration) on human path integration using a novel motion-cueing algorithm. Sensory modality and control dynamics were both varied randomly across trials, as participants controlled a joystick to steer to a memorized target location in virtual reality. Visual and vestibular steering cues allowed comparable accuracies only when participants controlled their acceleration, suggesting that vestibular signals, on their own, fail to support accurate path integration in the absence of sustained acceleration. Nevertheless, performance in all conditions reflected a failure to fully adapt to changes in the underlying control dynamics, a result that was well explained by a bias in the dynamics estimation. This work demonstrates how an incorrect internal model of control dynamics affects navigation in volatile environments in spite of continuous sensory feedback.
Neuroscience studies in non-human primates (NHP) often follow the rule of thumb that results observed in one animal must be replicated in at least one other. However, we lack a statistical justification for this rule of thumb, or an analysis of whether including three or more animals is better than including two. Yet, a formal statistical framework for experiments with few subjects would be crucial for experimental design, ethical justification, and data analysis. Also, including three or four animals in a study creates the possibility that the results observed in one animal will differ from those observed in the others: we need a statistically justified rule to resolve such situations. Here, I present a statistical framework to address these issues. This framework assumes that conducting an experiment will produce a similar result for a large proportion of the population (termed ‘representative’), but will produce spurious results for a substantial proportion of animals (termed ‘outliers’); the fractions of ‘representative’ and ‘outliers’ animals being defined by a prior distribution. I propose a procedure in which experimenters collect results from M animals and accept results that are observed in at least N of them (‘N-out-of-M’ procedure). I show how to compute the risks α (of reaching an incorrect conclusion) and β (of failing to reach a conclusion) for any prior distribution, and as a function of N and M. Strikingly, I find that the N-out-of-M model leads to a similar conclusion across a wide range of prior distributions: recordings from two animals lowers the risk α and therefore ensures reliable result, but leaves a large risk β; and recordings from three animals and accepting results observed in two of them strikes an efficient balance between acceptable risks α and β. This framework gives a formal justification for the rule of thumb of using at least two animals in NHP studies, suggests that recording from three animals when possible markedly improves statistical power, provides a statistical solution for situations where results are not consistent between all animals, and may apply to other types of studies involving few animals.
The neural mechanisms that unfold when humans form a large group defined by an overarching context, such as audiences in theater or sports, are largely unknown and unexplored. This is mainly due to the lack of availability of a scalable system that can record the brain activity from a significantly large portion of such an audience simultaneously. Although the technology for such a system has been readily available for a long time, the high cost as well as the large overhead in human resources and logistic planning have prohibited the development of such a system. However, during the recent years reduction in technology costs and size have led to the emergence of low-cost, consumer-oriented EEG systems, developed primarily for recreational use. Here by combining such a low-cost EEG system with other off-the-shelve hardware and tailor-made software, we develop in the lab and test in a cinema such a scalable EEG hyper-scanning system. The system has a robust and stable performance and achieves accurate unambiguous alignment of the recorded data of the different EEG headsets. These characteristics combined with small preparation time and low-cost make it an ideal candidate for recording large portions of audiences.
Research on psychopathy has so far been largely limited to the investigation of high-level processes, such as emotion perception and regulation. In the present work, we investigate whether psychopathy has an effect on the estimation of fundamental physical parameters, which are computed in the brain during early stages of sensory processing. We employed a simple task in which participants had to estimate their interpersonal distance from a moving avatar and stop it at a given distance. The face expression of the avatars were positive, negative, or neutral. Participants carried out the task online on their home computers. We measured the psychopathy level via a self-report questionnaire. Regardless of the degree of psychopathy, the facial expression of the avatars showed no effect on distance estimation. Our results show that individuals with a high degree of psychopathy underestimate distance of approaching avatars significantly less (let the avatar approach them significantly closer) than did participants with a lesser degree of psychopathy. Moreover, participants who scored high in Self-Centered Impulsivity underestimate the distance to approaching avatars significantly less (let the avatar approach closer) than participants with a low score. Distance estimation is considered an automatic process performed at early stages of visual processing. Therefore, our results imply that psychopathy affects basic early sensory processes, such as feature extraction, in the visual cortex.
Moving in synchrony to external rhythmic stimuli is an elementary function that humans regularly engage in. It is termed “sensorimotor synchronization” and it is governed by two main parameters, the period and the phase of the movement with respect to the external rhythm. There has been an extensive body of research on the characteristics of these parameters, primarily once the movement synchronization has reached a steady-state level. Particular interest has been shown about how these parameters are corrected when there are deviations for the steady-state level. However, little is known about the initial “tuning-in” interval, when one aligns the movement to the external rhythm from rest. The current work investigates this “tuning-in” period for each of the four limbs and makes various novel contributions in the understanding of sensorimotor synchronization. The results suggest that phase and period alignment appear to be separate processes. Phase alignment involves limb-specific somatosensory memory in the order of minutes while period alignment has very limited memory usage. Phase alignment is the primary task but then the brain switches to period alignment where it spends most its resources. In overall this work suggests a central, cognitive role of period alignment and a peripheral, sensorimotor role of phase alignment.
Temporal anticipation is a fundamental process underlying complex neural functions such as associative learning, decision-making, and motor-preparation. Here we study event anticipation in its simplest form in human participants using magnetoencephalography. We distributed events in time according to different probability density functions and presented the stimuli separately in two different sensory modalities. We found that the temporal dynamics in right parietal cortex correlate with reaction times to anticipated events. Specifically, after an event occurred, event probability was represented in right parietal activity, hinting at a functional role of event-related potential component P300 in temporal expectancy. The results are consistent across both visual and auditory modalities. The right parietal cortex seems to play a central role in the processing of event probability density. Overall, this work contributes to the understanding of the neural processes involved in the anticipation of events in time.
Models of perceptual decision making have historically been designed to maximally explain behaviour and brain activity independently of their ability to actually perform tasks. More recently, performance-optimized models have been shown to correlate with brain responses to images and thus present a complementary approach to understand perceptual processes. In the present study, we compare how these approaches comparatively account for the spatio-temporal organization of neural responses elicited by ambiguous visual stimuli. Forty-six healthy human subjects performed perceptual decisions on briefly flashed stimuli constructed from ambiguous characters. The stimuli were designed to have 7 orthogonal properties, ranging from low-sensory levels (e.g. spatial location of the stimulus) to conceptual (whether stimulus is a letter or a digit) and task levels (i.e. required hand movement). Magneto-encephalography source and decoding analyses revealed that these 7 levels of representations are sequentially encoded by the cortical hierarchy, and actively maintained until the subject responds. This hierarchy appeared poorly correlated to normative, drift-diffusion, and 5-layer convolutional neural networks (CNN) optimized to accurately categorize alpha-numeric characters, but partially matched the sequence of activations of 3/6 state-of-the-art CNNs trained for natural image labeling (VGG-16, VGG-19, MobileNet). Additionally, we identify several systematic discrepancies between these CNNs and brain activity, revealing the importance of single-trial learning and recurrent processing. Overall, our results strengthen the notion that performance-optimized algorithms can converge towards the computational solution implemented by the human visual system, and open possible avenues to improve artificial perceptual decision making.