TY  - JOUR
A1  - Dwivedi, Kshitij
A1  - Bonner, Michael F.
A1  - Cichy, Radoslaw Martin
A1  - Roig Noguera, Gemma
T1  - Unveiling functions of the visual cortex using task-specific deep neural networks
T2  - PLoS Computational Biology
N2  - Abstract: The human visual cortex enables visual perception through a cascade of hierarchical computations in cortical regions with distinct functionalities. Here, we introduce an AI-driven approach to discover the functional mapping of the visual cortex. We related human brain responses to scene images measured with functional MRI (fMRI) systematically to a diverse set of deep neural networks (DNNs) optimized to perform different scene perception tasks. We found a structured mapping between DNN tasks and brain regions along the ventral and dorsal visual streams. Low-level visual tasks mapped onto early brain regions, 3-dimensional scene perception tasks mapped onto the dorsal stream, and semantic tasks mapped onto the ventral stream. This mapping was of high fidelity, with more than 60% of the explainable variance in nine key regions being explained. Together, our results provide a novel functional mapping of the human visual cortex and demonstrate the power of the computational approach.
Author Summary: Human visual perception is a complex cognitive feat known to be mediated by distinct cortical regions of the brain. However, the exact function of these regions remains unknown, and thus it remains unclear how those regions together orchestrate visual perception. Here, we apply an AI-driven brain mapping approach to reveal visual brain function. This approach integrates multiple artificial deep neural networks trained on a diverse set of functions with functional recordings of the whole human brain. Our results reveal a systematic tiling of visual cortex by mapping regions to particular functions of the deep networks. Together this constitutes a comprehensive account of the functions of the distinct cortical regions of the brain that mediate human visual perception.
KW  - Functional magnetic resonance imaging
KW  - Visual cortex
KW  - Vision
KW  - Sensory perception
KW  - Permutation
KW  - Semantics
KW  - Linear regression analysis
KW  - Neural networks
Y1  - 2021
UR  - http://publikationen.ub.uni-frankfurt.de/frontdoor/index/index/docId/64439
UR  - https://nbn-resolving.org/urn:nbn:de:hebis:30:3-644399
SN  - 1553-7358
N1  - G.R. thanks the support of the Alfons and Gertrud Kassel Foundation. R.M.C. is supported by DFG grants (CI241/1-1, CI241/3-1) and the ERC Starting Grant (ERC-2018- StG 803370). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
VL  - 17
IS  - 8, art. e1009267
SP  - 1
EP  - 22
PB  - Public Library of Science
CY  - San Francisco, Calif.
ER  -