Approximation and interpolation with neural network

Research Article
Open access

Approximation and interpolation with neural network

Yiming Yang 1*
  • 1 Fudan University    
  • *corresponding author 19300180068@fudan.edu.cn
Published on 8 December 2023 | https://doi.org/10.54254/2753-8818/18/20230354
TNS Vol.18
ISSN (Print): 2753-8826
ISSN (Online): 2753-8818
ISBN (Print): 978-1-83558-201-5
ISBN (Online): 978-1-83558-202-2

Abstract

In this paper we show that multilayer feedforward networks with one single hidden layer.and certain types of activation functions can approximate univariant continuous functions defined on a compact set. arbitrarily well. In particular, our results contain some usual activation functions such as sigmoidal functions, RELU functions and threshold functions. Besides, since interpolation problems are highly related to approximation problem, we demonstrate that a wide range of functions have the ability to interpolate and generalize our results to functions which are not polynomial on R. Compared to existing results by numerous work, our methods are more intuitive and less technical. Lastly, the paper discusses the possibility of combining interpolation property and approximating property together, and demonstrates that given any Riemann integrable functions on a compact set in R, with several points on its graph, the finite combination of monotone sigmoidal functions can pass through these points and approximate the given function arbitrarily well with respect to L^1 (dx) (in the sense of Riemann integral) when the number of points getting large.

Keywords:

neural networks, approximation, interpolation

Yang,Y. (2023). Approximation and interpolation with neural network. Theoretical and Natural Science,18,126-132.
Export citation

References

[1]. Cybenko,G., (1989) Approximation by superpositions of a sigmoidal function, Math Control Signals Systems 2, 303-314

[2]. Cotter,A.E. (1990) Stone-Weierstrass theorem and its application to neural networks. IEEE Trans. Neural Networks 1, 290-295

[3]. Hornik,K, (1991) Approximation capabilities of multilayers feedforword networks, Neural Network 1, 261-257

[4]. Leshno,M. Lin,V.Y., Pinkus,A. and Schocken,S. (1993), Multilayer feedforward neural networks with a non-polynomial activation function can approximate any function, Neural Networks 6, 861-867

[5]. Itô,Y. Saito,K. (1996) Superposition of linearly independent functions and finite mappings by neural networks, Math Scient 21, 27-33

[6]. Huang, G.B., Babri, H.A. (1998) Upper bounds on the number of hidden neurons in feedforward networks with arbitrary bounded nonlinear activation functions, IEEE Trans. Neural Networks 9, 224-229

[7]. Pinkus, A. (1999) Approximation theory of the MLP model in neural networks, Acta Numerica, 143-195

[8]. Grafakus. L.(2014) Classical Fourier Analysis (3rd ed.) Springer-Verlag, New York

[9]. Varga, R.S. (2002) Matrix Iterative Analysis (2nd ed.), Springer-Verlag, Berlin

[10]. Schwartz, L. (1947) Theorie Generale des Fonctions Moyenne-Periodiques.,Ann. Math. 48, 857-928


Cite this article

Yang,Y. (2023). Approximation and interpolation with neural network. Theoretical and Natural Science,18,126-132.

Data availability

The datasets used and/or analyzed during the current study will be available from the authors upon reasonable request.

Disclaimer/Publisher's Note

The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of EWA Publishing and/or the editor(s). EWA Publishing and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

About volume

Volume title: Proceedings of the 2nd International Conference on Computing Innovation and Applied Physics

ISBN:978-1-83558-201-5(Print) / 978-1-83558-202-2(Online)
Editor:Marwan Omar, Roman Bauer
Conference website: https://www.confciap.org/
Conference date: 25 March 2023
Series: Theoretical and Natural Science
Volume number: Vol.18
ISSN:2753-8818(Print) / 2753-8826(Online)

© 2024 by the author(s). Licensee EWA Publishing, Oxford, UK. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license. Authors who publish this series agree to the following terms:
1. Authors retain copyright and grant the series right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this series.
2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the series's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this series.
3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See Open access policy for details).

References

[1]. Cybenko,G., (1989) Approximation by superpositions of a sigmoidal function, Math Control Signals Systems 2, 303-314

[2]. Cotter,A.E. (1990) Stone-Weierstrass theorem and its application to neural networks. IEEE Trans. Neural Networks 1, 290-295

[3]. Hornik,K, (1991) Approximation capabilities of multilayers feedforword networks, Neural Network 1, 261-257

[4]. Leshno,M. Lin,V.Y., Pinkus,A. and Schocken,S. (1993), Multilayer feedforward neural networks with a non-polynomial activation function can approximate any function, Neural Networks 6, 861-867

[5]. Itô,Y. Saito,K. (1996) Superposition of linearly independent functions and finite mappings by neural networks, Math Scient 21, 27-33

[6]. Huang, G.B., Babri, H.A. (1998) Upper bounds on the number of hidden neurons in feedforward networks with arbitrary bounded nonlinear activation functions, IEEE Trans. Neural Networks 9, 224-229

[7]. Pinkus, A. (1999) Approximation theory of the MLP model in neural networks, Acta Numerica, 143-195

[8]. Grafakus. L.(2014) Classical Fourier Analysis (3rd ed.) Springer-Verlag, New York

[9]. Varga, R.S. (2002) Matrix Iterative Analysis (2nd ed.), Springer-Verlag, Berlin

[10]. Schwartz, L. (1947) Theorie Generale des Fonctions Moyenne-Periodiques.,Ann. Math. 48, 857-928