Refine
Language
- English (10)
Has Fulltext
- yes (10)
Is part of the Bibliography
- no (10)
Keywords
- EEG (2)
- Brain size I (1)
- Computational modelling (1)
- Gray matter volume (1)
- Handwriting (1)
- Intelligence (1)
- MVPA (1)
- Machine learning (1)
- N400 (1)
- Prediction (1)
Institute
- Psychologie (10)
- MPI für Hirnforschung (8)
- MPI für empirische Ästhetik (1)
How is semantic information stored in the human mind and brain? Some philosophers and cognitive scientists argue for vectorial representations of concepts, where the meaning of a word is represented as its position in a high-dimensional neural state space. At the intersection of natural language processing and artificial intelligence, a class of very successful distributional word vector models has developed that can account for classic EEG findings of language, that is, the ease versus difficulty of integrating a word with its sentence context. However, models of semantics have to account not only for context-based word processing, but should also describe how word meaning is represented. Here, we investigate whether distributional vector representations of word meaning can model brain activity induced by words presented without context. Using EEG activity (event-related brain potentials) collected while participants in two experiments (English and German) read isolated words, we encoded and decoded word vectors taken from the family of prediction-based Word2vec algorithms. We found that, first, the position of a word in vector space allows the prediction of the pattern of corresponding neural activity over time, in particular during a time window of 300 to 500 ms after word onset. Second, distributional models perform better than a human-created taxonomic baseline model (WordNet), and this holds for several distinct vector-based models. Third, multiple latent semantic dimensions of word meaning can be decoded from brain activity. Combined, these results suggest that empiricist, prediction-based vectorial representations of meaning are a viable candidate for the representational architecture of human semantic knowledge.
A positive association between brain size and intelligence is firmly established, but whether region-specific anatomical differences contribute to general intelligence remains an open question. Results from voxel-based morphometry (VBM) - one of the most widely used morphometric methods - have remained inconclusive so far. Here, we applied cross-validated machine learning-based predictive modeling to test whether out-of-sample prediction of individual intelligence scores is possible on the basis of voxel-wise gray matter volume. Features were derived from structural magnetic resonance imaging data (N = 308) using (a) a purely data-driven method (principal component analysis) and (b) a domain knowledge-based approach (atlas parcellation). When using relative gray matter (corrected for total brain size), only the atlas-based approach provided significant prediction, while absolute gray matter (uncorrected) allowed for above-chance prediction with both approaches. Importantly, in all significant predictions, the absolute error was relatively high, i.e., greater than ten IQ points, and in the atlas-based models, the predicted IQ scores varied closely around the sample mean. This renders the practical value even of statistically significant prediction results questionable. Analyses based on the gray matter of functional brain networks yielded significant predictions for the fronto-parietal network and the cerebellum. However, the mean absolute errors were not reduced in contrast to the global models, suggesting that general intelligence may be related more to global than region-specific differences in gray matter volume. More generally, our study highlights the importance of predictive statistical analysis approaches for clarifying the neurobiological bases of intelligence and provides important suggestions for future research using predictive modeling.
Children with attention-deficit/hyperactivity disorder (ADHD) are characterized by symptoms of inattention, impulsivity, and hyperactivity. Neurophysiological correlates of ADHD include changes in the P3 component of event-related brain potentials (ERPs). Motivated by recent advances towards a more dimensional understanding of ADHD, we investigate whether ADHD-related ERP markers relate to continuous variations in attention and executive functioning also in typically-developing children. ERPs were measured while 31 school children (9–11 years) completed an adapted version of the Continuous Performance Task that additionally to inhibitory processes also isolates effects of physical stimulus salience. Children with higher levels of parent-reported ADHD symptoms did not differ in task performance, but exhibited smaller P3 amplitudes related to stimulus salience. Furthermore, ADHD symptoms were associated with the variability of neural responses over time: Children with higher levels of ADHD symptoms demonstrated lower variability in inhibition- and salience-related P3 amplitudes. No effects were observed for ERP latencies and the salience-related N2. By demonstrating that ADHD-associated neurophysiological mechanisms of inhibition and salience processing covary with attention and executive functioning in a children community sample, our study provides neurophysiological support for dimensional models of ADHD. Also, temporal variability in event-related potentials is highlighted as additional indicator of ADHD requiring further investigation.
Most current models assume that the perceptual and cognitive processes of visual word recognition and reading operate upon neuronally coded domain-general low-level visual representations – typically oriented line representations. We here demonstrate, consistent with neurophysiological theories of Bayesian-like predictive neural computations, that prior visual knowledge of words may be utilized to ‘explain away’ redundant and highly expected parts of the visual percept. Subsequent processing stages, accordingly, operate upon an optimized representation of the visual input, the orthographic prediction error, highlighting only the visual information relevant for word identification. We show that this optimized representation is related to orthographic word characteristics, accounts for word recognition behavior, and is processed early in the visual processing stream, i.e., in V4 and before 200 ms after word-onset. Based on these findings, we propose that prior visual-orthographic knowledge is used to optimize the representation of visually presented words, which in turn allows for highly efficient reading processes.
Most current models assume that the perceptual and cognitive processes of visual word recognition and reading operate upon neuronally coded domain-general low-level visual representations – typically oriented line representations. We here demonstrate, consistent with neurophysiological theories of Bayesian-like predictive neural computations, that prior visual knowledge of words may be utilized to ‘explain away’ redundant and highly expected parts of the visual percept. Subsequent processing stages, accordingly, operate upon an optimized representation of the visual input, the orthographic prediction error, highlighting only the visual information relevant for word identification. We show that this optimized representation is related to orthographic word characteristics, accounts for word recognition behavior, and is processed early in the visual processing stream, i.e., in V4 and before 200 ms after word-onset. Based on these findings, we propose that prior visual-orthographic knowledge is used to optimize the representation of visually presented words, which in turn allows for highly efficient reading processes.
Abstract
To characterize the functional role of the left-ventral occipito-temporal cortex (lvOT) during reading in a quantitatively explicit and testable manner, we propose the lexical categorization model (LCM). The LCM assumes that lvOT optimizes linguistic processing by allowing fast meaning access when words are familiar and filtering out orthographic strings without meaning. The LCM successfully simulates benchmark results from functional brain imaging described in the literature. In a second evaluation, we empirically demonstrate that quantitative LCM simulations predict lvOT activation better than alternative models across three functional magnetic resonance imaging studies. We found that word-likeness, assumed as input into a lexical categorization process, is represented posteriorly to lvOT, whereas a dichotomous word/non-word output of the LCM could be localized to the downstream frontal brain regions. Finally, training the process of lexical categorization resulted in more efficient reading. In sum, we propose that word recognition in the ventral visual stream involves word-likeness extraction followed by lexical categorization before one can access word meaning.
Author summary
Visual word recognition is a critical process for reading and relies on the human brain’s left ventral occipito-temporal (lvOT) regions. However, the lvOTs specific function in visual word recognition is not yet clear. We propose that these occipito-temporal brain systems are critical for lexical categorization, i.e., the process of determining whether an orthographic percept is a known word or not, so that further lexical and semantic processing can be restricted to those percepts that are part of our "mental lexicon". We demonstrate that a computational model implementing this process, the lexical categorization model, can explain seemingly contradictory benchmark results from the published literature. We further use functional magnetic resonance imaging to show that the lexical categorization model successfully predicts brain activation in the left ventral occipito-temporal cortex elicited during a word recognition task. It does so better than alternative models proposed so far. Finally, we provide causal evidence supporting this model by empirically demonstrating that training the process of lexical categorization improves reading performance.
To characterize the left-ventral occipito-temporal cortex (lvOT) role during reading in a quantitatively explicit and testable manner, we propose the lexical categorization model (LCM). The LCM assumes that lvOT optimizes linguistic processing by allowing fast meaning access when words are familiar and filter out orthographic strings without meaning. The LCM successfully simulates benchmark results from functional brain imaging. Empirically, using functional magnetic resonance imaging, we demonstrate that quantitative LCM simulations predict lvOT activation across three studies better than alternative models. Besides, we found that word-likeness, which is assumed as input to LCM, is represented posterior to lvOT. In contrast, a dichotomous word/non-word contrast, which is assumed as the LCM’s output, could be localized to upstream frontal brain regions. Finally, we found that training lexical categorization results in more efficient reading. Thus, we propose a ventral-visual-stream processing framework for reading involving word-likeness extraction followed by lexical categorization, before meaning extraction.
Across languages, the speech signal is characterized by a predominant modulation of the amplitude spectrum between about 4.3-5.5Hz, reflecting the production and processing of linguistic information chunks (syllables, words) every ∼200ms. Interestingly, ∼200ms is also the typical duration of eye fixations during reading. Prompted by this observation, we demonstrate that German readers sample written text at ∼5Hz. A subsequent meta-analysis with 142 studies from 14 languages replicates this result, but also shows that sampling frequencies vary across languages between 3.9Hz and 5.2Hz, and that this variation systematically depends on the complexity of the writing systems (character-based vs. alphabetic systems, orthographic transparency). Finally, we demonstrate empirically a positive correlation between speech spectrum and eye-movement sampling in low-skilled readers. Based on this convergent evidence, we propose that during reading, our brain’s linguistic processing systems imprint a preferred processing rate, i.e., the rate of spoken language production and perception, onto the oculomotor system.
How is semantic information stored in the human mind and brain? Some philosophers and cognitive scientists argue for vectorial representations of concepts, where the meaning of a word is represented as its position in a high-dimensional neural state space. At the intersection of natural language processing and artificial intelligence, a class of very successful distributional word vector models has developed that can account for classic EEG findings of language, i.e., the ease vs. difficulty of integrating a word with its sentence context. However, models of semantics have to account not only for context-based word processing, but should also describe how word meaning is represented. Here, we investigate whether distributional vector representations of word meaning can model brain activity induced by words presented without context. Using EEG activity (event-related brain potentials) collected while participants in two experiments (English, German) read isolated words, we encode and decode word vectors taken from the family of prediction-based word2vec algorithms. We find that, first, the position of a word in vector space allows the prediction of the pattern of corresponding neural activity over time, in particular during a time window of 300 to 500 ms after word onset. Second, distributional models perform better than a human-created taxonomic baseline model (WordNet), and this holds for several distinct vector-based models. Third, multiple latent semantic dimensions of word meaning can be decoded from brain activity. Combined, these results suggest that empiricist, prediction-based vectorial representations of meaning are a viable candidate for the representational architecture of human semantic knowledge.
The outstanding speed of language comprehension necessitates a highly efficient implementation of cognitive-linguistic processes. The domain-general theory of Predictive Coding suggests that our brain solves this problem by continuously forming linguistic predictions about expected upcoming input. The neurophysiological implementation of these predictive linguistic processes, however, is not yet understood. Here, we use EEG (human participants, both sexes) to investigate the existence and nature of online-generated, category-level semantic representations during sentence processing. We conducted two experiments in which some nouns – embedded in a predictive spoken sentence context – were unexpectedly delayed by 1 second. Target nouns were either abstract/concrete (Experiment 1) or animate/inanimate (Experiment 2). We hypothesized that if neural prediction error signals following (temporary) omissions carry specific information about the stimulus, the semantic category of the upcoming target word is encoded in brain activity prior to its presentation. Using time-generalized multivariate pattern analysis, we demonstrate significant decoding of word category from silent periods directly preceding the target word, in both experiments. This provides direct evidence for predictive coding during sentence processing, i.e., that information about a word can be encoded in brain activity before it is perceived. While the same semantic contrast could also be decoded from EEG activity elicited by isolated words (Experiment 1), the identified neural patterns did not generalize to pre-stimulus delay period activity in sentences. Our results not only indicate that the brain processes language predictively, but also demonstrate the nature and sentence-specificity of category-level semantic predictions preactivated during sentence comprehension.