Learning abstract representations through lossy compression of multi-modal signals
- A key competence for open-ended learning is the formation of increasingly abstract representations useful for driving complex behavior. Abstract representations ignore specific details and facilitate generalization. Here we consider the learning of abstract representations in a multi-modal setting with two or more input modalities. We treat the problem as a lossy compression problem and show that generic lossy compression of multimodal sensory input naturally extracts abstract representations that tend to strip away modalitiy specific details and preferentially retain information that is shared across the different modalities. Furthermore, we propose an architecture to learn abstract representations by identifying and retaining only the information that is shared across multiple modalities while discarding any modality specific information.
| Author: | Charles WilmotORCiDGND, Gianluca BaldassarreORCiDGND, Jochen TrieschORCiD |
|---|---|
| URN: | urn:nbn:de:hebis:30:3-735307 |
| DOI: | https://doi.org/10.48550/arXiv.2101.11376 |
| ArXiv Id: | http://arxiv.org/abs/2101.11376v3 |
| Document Type: | Preprint |
| Language: | English |
| Year of Completion: | 2021 |
| Year of first Publication: | 2021 |
| Publishing Institution: | Universitätsbibliothek Johann Christian Senckenberg |
| Release Date: | 2023/07/06 |
| Page Number: | 14 |
| HeBIS-PPN: | 510536786 |
| Institutes: | Informatik und Mathematik / Informatik |
| Wissenschaftliche Zentren und koordinierte Programme / Frankfurt Institute for Advanced Studies (FIAS) | |
| Dewey Decimal Classification: | 0 Informatik, Informationswissenschaft, allgemeine Werke / 00 Informatik, Wissen, Systeme / 004 Datenverarbeitung; Informatik |
| Sammlungen: | Universitätspublikationen |
| Licence (German): |  Creative Commons - CC BY-SA - Namensnennung - Weitergabe unter gleichen Bedingungen 4.0 International |
