Approximator networks and the principle of optimal information distribution

  • It is well known that artificial neural nets can be used as approximators of any continous functions to any desired degree. Nevertheless, for a given application and a given network architecture the non-trivial task rests to determine the necessary number of neurons and the necessary accuracy (number of bits) per weight for a satisfactory operation. In this paper the problem is treated by an information theoretic approach. The values for the weights and thresholds in the approximator network are determined analytically. Furthermore, the accuracy of the weights and the number of neurons are seen as general system parameters which determine the the maximal output information (i.e. the approximation error) by the absolute amount and the relative distribution of information contained in the network. A new principle of optimal information distribution is proposed and the conditions for the optimal system parameters are derived. For the simple, instructive example of a linear approximation of a non-linear, quadratic function, the principle of optimal information distribution gives the the optimal system parameters, i.e. the number of neurons and the different resolutions of the variables.

Download full text files

Export metadata

Additional Services

Share in Twitter Search Google Scholar
Metadaten
Author:Rüdiger W. BrauseGND
URN:urn:nbn:de:hebis:30-79017
Parent Title (German):Universität Frankfurt am Main. Fachbereich Informatik: Interner Bericht ; 91,1
Series (Serial Number):Interner Bericht / Fachbereich Informatik, Johann Wolfgang Goethe-Universität Frankfurt a.M. (91,1)
Document Type:Working Paper
Language:English
Year of Completion:1991
Year of first Publication:1991
Publishing Institution:Universitätsbibliothek Johann Christian Senckenberg
Release Date:2010/09/08
HeBIS-PPN:227673492
Institutes:Informatik und Mathematik / Informatik
Dewey Decimal Classification:0 Informatik, Informationswissenschaft, allgemeine Werke / 00 Informatik, Wissen, Systeme / 004 Datenverarbeitung; Informatik
Sammlungen:Universitätspublikationen
Licence (German):License LogoDeutsches Urheberrecht