The Fisher information as a neural guiding principle for independent component analysis

The Fisher information constitutes a natural measure for the sensitivity of a probability distribution with respect to a set of parameters. An implementation of the stationarity principle for synaptic learning in terms o
The Fisher information constitutes a natural measure for the sensitivity of a probability distribution with respect to a set of parameters. An implementation of the stationarity principle for synaptic learning in terms of the Fisher information results in a Hebbian self-limiting learning rule for synaptic plasticity. In the present work, we study the dependence of the solutions to this rule in terms of the moments of the input probability distribution and find a preference for non-Gaussian directions, making it a suitable candidate for independent component analysis (ICA). We confirm in a numerical experiment that a neuron trained under these rules is able to find the independent components in the non-linear bars problem. The specific form of the plasticity rule depends on the transfer function used, becoming a simple cubic polynomial of the membrane potential for the case of the rescaled error function. The cubic learning rule is also an excellent approximation for other transfer functions, as the standard sigmoidal, and can be used to show analytically that the proposed plasticity rules are selective for directions in the space of presynaptic neural activities characterized by a negative excess kurtosis.
…show moreshow less

Download full text files

Export metadata

  • Export Bibtex
  • Export RIS

Additional Services

    Share in Twitter Search Google Scholar
Metadaten
Author:Rodrigo Echeveste, Samuel Eckmann, Claudius Gros
URN:urn:nbn:de:hebis:30:3-402660
DOI:http://dx.doi.org/10.3390/e17063838
ISSN:1099-4300
Parent Title (English):Entropy
Publisher:MDPI
Place of publication:Basel
Document Type:Article
Language:English
Date of Publication (online):2015/06/09
Date of first Publication:2015/06/09
Publishing Institution:Universitätsbibliothek Johann Christian Senckenberg
Release Date:2016/05/30
Tag:Fisher information; Hebbian learning; excess kurtosis; guiding principle; independent component analysis; objective functions; synaptic plasticity
Issue:6
Pagenumber:19
First Page:3838
Last Page:3856
Note:
c 2015 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/).
HeBIS PPN:428604528
Institutes:Physik
Dewey Decimal Classification:004 Datenverarbeitung; Informatik
510 Mathematik
Sammlungen:Universitätspublikationen
Licence (German):License LogoCreative Commons - Namensnennung 4.0

$Rev: 11761 $