BOLD Moments: modeling short visual events through a video fMRI dataset and metadata

  • Grasping the meaning of everyday visual events is a fundamental feat of human intelligence that hinges on diverse neural processes ranging from vision to higher-level cognition. Deciphering the neural basis of visual event understanding requires rich, extensive, and appropriately designed experimental data. However, this type of data is hitherto missing. To fill this gap, we introduce the BOLD Moments Dataset (BMD), a large dataset of whole-brain fMRI responses to over 1,000 short (3s) naturalistic video clips and accompanying metadata. We show visual events interface with an array of processes, extending even to memory, and we reveal a match in hierarchical processing between brains and video-computable deep neural networks. Furthermore, we showcase that BMD successfully captures temporal dynamics of visual events at second resolution. BMD thus establishes a critical groundwork for investigations of the neural basis of visual event understanding.

Download full text files

Export metadata

Author:Benjamin LahnerORCiD, Kshitij DwivediORCiDGND, Polina IamshchininaORCiDGND, Monika GraumannORCiD, Alex LascellesORCiD, Gemma Roig NogueraORCiDGND, Alessandro Thomas GiffordORCiD, Bowen Pan, SouYoung JinORCiD, N. Apurva Ratan MurtyORCiD, Kendrick KayORCiD, Aude Oliva, Radoslaw Martin CichyORCiDGND
Parent Title (English):bioRxiv
Document Type:Preprint
Date of Publication (online):2023/03/13
Date of first Publication:2023/03/13
Publishing Institution:Universit├Ątsbibliothek Johann Christian Senckenberg
Release Date:2023/03/27
Page Number:71
Institutes:Informatik und Mathematik / Informatik
Dewey Decimal Classification:0 Informatik, Informationswissenschaft, allgemeine Werke / 00 Informatik, Wissen, Systeme / 004 Datenverarbeitung; Informatik
1 Philosophie und Psychologie / 15 Psychologie / 150 Psychologie
Licence (German):License LogoCreative Commons - CC BY-NC-ND - Namensnennung - Nicht kommerziell - Keine Bearbeitungen 4.0 International