Local homeostatic regulation of the spectral radius of echo-state networks

  • Recurrent cortical networks provide reservoirs of states that are thought to play a crucial role for sequential information processing in the brain. However, classical reservoir computing requires manual adjustments of global network parameters, particularly of the spectral radius of the recurrent synaptic weight matrix. It is hence not clear if the spectral radius is accessible to biological neural networks. Using random matrix theory, we show that the spectral radius is related to local properties of the neuronal dynamics whenever the overall dynamical state is only weakly correlated. This result allows us to introduce two local homeostatic synaptic scaling mechanisms, termed flow control and variance control, that implicitly drive the spectral radius toward the desired value. For both mechanisms the spectral radius is autonomously adapted while the network receives and processes inputs under working conditions. We demonstrate the effectiveness of the two adaptation mechanisms under different external input protocols. Moreover, we evaluated the network performance after adaptation by training the network to perform a time-delayed XOR operation on binary sequences. As our main result, we found that flow control reliably regulates the spectral radius for different types of input statistics. Precise tuning is however negatively affected when interneural correlations are substantial. Furthermore, we found a consistent task performance over a wide range of input strengths/variances. Variance control did however not yield the desired spectral radii with the same precision, being less consistent across different input strengths. Given the effectiveness and remarkably simple mathematical form of flow control, we conclude that self-consistent local control of the spectral radius via an implicit adaptation scheme is an interesting and biological plausible alternative to conventional methods using set point homeostatic feedback controls of neural firing.
Author:Fabian SchubertORCiDGND, Claudius GrosORCiDGND
Parent Title (English):Frontiers in computational neuroscience
Publisher:Frontiers Research Foundation
Place of publication:Lausanne
Document Type:Article
Date of Publication (online):2021/02/24
Date of first Publication:2021/02/24
Publishing Institution:Universitätsbibliothek Johann Christian Senckenberg
Release Date:2021/05/19
Tag:echo-state networks; homeostasis; recurrent networks; reservoir computing; spectral radius; synaptic scaling
Issue:art. 587721
Page Number:17
First Page:1
Last Page:17
Copyright © 2021 Schubert and Gros. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
Dewey Decimal Classification:5 Naturwissenschaften und Mathematik / 53 Physik / 530 Physik
Licence (German):License LogoCreative Commons - Namensnennung 4.0