Refine
Document Type
- Article (3)
- Doctoral Thesis (1)
Language
- English (4)
Has Fulltext
- yes (4)
Is part of the Bibliography
- no (4)
Keywords
- Active learning (1)
- Congenital ocular motor apraxia (1)
- Continual deep learning (1)
- Joubert syndrome (1)
- Lifelong machine learning (1)
- Molar tooth sign (1)
- Open set recognition (1)
- Open world learning (1)
- catastrophic forgetting (1)
- continual deep learning (1)
- deep generative models (1)
- open-set recognition (1)
- variational inference (1)
Institute
- Informatik und Mathematik (2)
- Informatik (1)
- Medizin (1)
- Psychologie (1)
Nosological delineation of congenital ocular motor apraxia type Cogan : an observational study
(2016)
Background: The nosological assignment of congenital ocular motor apraxia type Cogan (COMA) is still controversial. While regarded as a distinct entity by some authorities including the Online Mendelian Inheritance in Man catalog of genetic disorders, others consider COMA merely a clinical symptom.
Methods: We performed a retrospective multicenter data collection study with re-evaluation of clinical and neuroimaging data of 21 previously unreported patients (8 female, 13 male, ages ranging from 2 to 24 years) diagnosed as having COMA.
Results: Ocular motor apraxia (OMA) was recognized during the first year of life and confined to horizontal pursuit in all patients. OMA attenuated over the years in most cases, regressed completely in two siblings, and persisted unimproved in one individual. Accompanying clinical features included early onset ataxia in most patients and cognitive impairment with learning disability (n = 6) or intellectual disability (n = 4). Re-evaluation of MRI data sets revealed a hitherto unrecognized molar tooth sign diagnostic for Joubert syndrome in 11 patients, neuroimaging features of Poretti-Boltshauser syndrome in one case and cerebral malformation suspicious of a tubulinopathy in another subject. In the remainder, MRI showed vermian hypo-/dysplasia in 4 and no abnormalities in another 4 patients. There was a strong trend to more severe cognitive impairment in patients with Joubert syndrome compared to those with inconclusive MRI, but otherwise no significant difference in clinical phenotypes between these two groups.
Conclusions: Systematical renewed analysis of neuroimaging data resulted in a diagnostic reappraisal in the majority of patients with early-onset OMA in the cohort reported here. This finding poses a further challenge to the notion of COMA constituting a separate entity and underlines the need for an expert assessment of neuroimaging in children with COMA, especially if they show cognitive impairment.
Deep learning with neural networks seems to have largely replaced traditional design of computer vision systems. Automated methods to learn a plethora of parameters are now used in favor of previously practiced selection of explicit mathematical operators for a specific task. The entailed promise is that practitioners no longer need to take care of every individual step, but rather focus on gathering big amounts of data for neural network training. As a consequence, both a shift in mindset towards a focus on big datasets, as well as a wave of conceivable applications based exclusively on deep learning can be observed.
This PhD dissertation aims to uncover some of the only implicitly mentioned or overlooked deep learning aspects, highlight unmentioned assumptions, and finally introduce methods to address respective immediate weaknesses. In the author’s humble opinion, these prevalent shortcomings can be tied to the fact that the involved steps in the machine learning workflow are frequently decoupled. Success is predominantly measured based on accuracy measures designed for evaluation with static benchmark test sets. Individual machine learning workflow components are assessed in isolation with respect to available data, choice of neural network architecture, and a particular learning algorithm, rather than viewing the machine learning system as a whole in context of a particular application. Correspondingly, in this dissertation, three key challenges have been identified: 1. Choice and flexibility of a neural network architecture. 2. Identification and rejection of unseen unknown data to avoid false predictions. 3. Continual learning without forgetting of already learned information. These latter challenges have already been crucial topics in older literature, alas, seem to require a renaissance in modern deep learning literature. Initially, it may appear that they pose independent research questions, however, the thesis posits that the aspects are intertwined and require a joint perspective in machine learning based systems. In summary, the essential question is thus how to pick a suitable neural network architecture for a specific task, how to recognize which data inputs belong to this context, which ones originate from potential other tasks, and ultimately how to continuously include such identified novel data in neural network training over time without overwriting existing knowledge.
Thus, the central emphasis of this dissertation is to build on top of existing deep learning strengths, yet also acknowledge mentioned weaknesses, in an effort to establish a deeper understanding of interdependencies and synergies towards the development of unified solution mechanisms. For this purpose, the main portion of the thesis is in cumulative form. The respective publications can be grouped according to the three challenges outlined above. Correspondingly, chapter 1 is focused on choice and extendability of neural network architectures, analyzed in context of popular image classification tasks. An algorithm to automatically determine neural network layer width is introduced and is first contrasted with static architectures found in the literature. The importance of neural architecture design is then further showcased on a real-world application of defect detection in concrete bridges. Chapter 2 is comprised of the complementary ensuing questions of how to identify unknown concepts and subsequently incorporate them into continual learning. A joint central mechanism to distinguish unseen concepts from what is known in classification tasks, while enabling consecutive training without forgetting or revisiting older classes, is proposed. Once more, the role of the chosen neural network architecture is quantitatively reassessed. Finally, chapter 3 culminates in an overarching view, where developed parts are connected. Here, an extensive survey further serves the purpose to embed the gained insights in the broader literature landscape and emphasizes the importance of a common frame of thought. The ultimately presented approach thus reflects the overall thesis’ contribution to advance neural network based machine learning towards a unified solution that ties together choice of neural architecture with the ability to learn continually and the capability to automatically separate known from unknown data.
Unified probabilistic deep continual learning through generative replay and open set recognition
(2022)
Modern deep neural networks are well known to be brittle in the face of unknown data instances and recognition of the latter remains a challenge. Although it is inevitable for continual-learning systems to encounter such unseen concepts, the corresponding literature appears to nonetheless focus primarily on alleviating catastrophic interference with learned representations. In this work, we introduce a probabilistic approach that connects these perspectives based on variational inference in a single deep autoencoder model. Specifically, we propose to bound the approximate posterior by fitting regions of high density on the basis of correctly classified data points. These bounds are shown to serve a dual purpose: unseen unknown out-of-distribution data can be distinguished from already trained known tasks towards robust application. Simultaneously, to retain already acquired knowledge, a generative replay process can be narrowed to strictly in-distribution samples, in order to significantly alleviate catastrophic interference.
Current deep learning methods are regarded as favorable if they empirically perform well on dedicated test sets. This mentality is seamlessly reflected in the resurfacing area of continual learning, where consecutively arriving data is investigated. The core challenge is framed as protecting previously acquired representations from being catastrophically forgotten. However, comparison of individual methods is nevertheless performed in isolation from the real world by monitoring accumulated benchmark test set performance. The closed world assumption remains predominant, i.e. models are evaluated on data that is guaranteed to originate from the same distribution as used for training. This poses a massive challenge as neural networks are well known to provide overconfident false predictions on unknown and corrupted instances. In this work we critically survey the literature and argue that notable lessons from open set recognition, identifying unknown examples outside of the observed set, and the adjacent field of active learning, querying data to maximize the expected performance gain, are frequently overlooked in the deep learning era. Hence, we propose a consolidated view to bridge continual learning, active learning and open set recognition in deep neural networks. Finally, the established synergies are supported empirically, showing joint improvement in alleviating catastrophic forgetting, querying data, selecting task orders, while exhibiting robust open world application.