Download pdf
An exploration of KANs and CKANs for more efficient deep learning architecture
- 1 DUT School of Software Technology & DUT-RU International School of Information Science and Engineering, Dalian University of Technology, Dalian, Liaoning, 116620, China
* Author to whom correspondence should be addressed.
Abstract
Deep learning has revolutionized the field of machine learning with its ability to discern complex patterns from voluminous data. Despite the success of Multi-Layer Perceptrons (MLPs) and Convolutional Neural Networks (CNNs), there is an ongoing quest for architectures that offer higher expressiveness with fewer parameters. This paper focuses on the Kolmogorov-Arnold Networks (KANs) and Convolutional Kolmogorov-Arnold Networks (CKANs), which integrate learnable spline functions for enhanced expressiveness and efficiency. This study designs a range of networks to compare KANs with MLPs and CKANs with classical CNNs on the CIFAR-10 dataset. Moreover, this study evaluates the models based on several metrics, including accuracy, precision, recall, F1 score, and parameter count. Based on the experimental results, networks with KANs and CKANs demonstrated improved accuracy with a reduced parameter footprint, indicating the potential of KAN-based models in capturing complex patterns. In conclusion, integrating KANs into CNNs and MLPs is a promising approach for developing more efficient and interpretable models, offering a path to advance deep learning architectures.
Keywords
Kolmogorov-Arnold network, Neural network, Convolutional neural network
[1]. Liu, Z., Wang, Y., Vaidya, S., Ruehle, F., Halverson, J., et, al. (2024). Kan: Kolmogorov-arnold networks. arXiv preprint arXiv:2404.19756.
[2]. Hornik, K., Stinchcombe, M., & White, H. (1989). Multilayer feedforward networks are universal approximators. Neural networks, 2(5), 359-366.
[3]. LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278-2324.
[4]. Li, Z., Liu, F., Yang, W., Peng, S., & Zhou, J. (2021). A survey of convolutional neural networks: analysis, applications, and prospects. IEEE transactions on neural networks and learning systems, 33(12), 6999-7019.
[5]. Bodner, A. D., Tepsich, A. S., Spolski, J. N., & Pourteau, S. (2024). Convolutional Kolmogorov-Arnold Networks. arXiv preprint arXiv:2406.13155.
[6]. Fey, M., Lenssen, J. E., Weichert, F., & Müller, H. (2018). Splinecnn: Fast geometric deep learning with continuous b-spline kernels. In Proceedings of the IEEE conference on computer vision and pattern recognition, 869-877.
[7]. Drokin, I. (2024). Kolmogorov-Arnold Convolutions: Design Principles and Empirical Studies. arXiv preprint arXiv:2407.01092.
[8]. Jamali, A., Roy, S. K., Hong, D., Lu, B., & Ghamisi, P. (2024). How to Learn More? Exploring Kolmogorov-Arnold Networks for Hyperspectral Image Classification. arXiv preprint arXiv:2406.15719.
[9]. Arnold, V. I. (2009). On functions of three variables. Collected Works: Representations of Functions, Celestial Mechanics and KAM Theory, 1957–1965, 5-8.
[10]. CIFAR-10 Dataset. URL: https://www.cs.toronto.edu/~kriz/cifar.html. Last Accessed: 2024/08/24
Cite this article
Zhou,S. (2024). An exploration of KANs and CKANs for more efficient deep learning architecture. Applied and Computational Engineering,83,20-25.
Data availability
The datasets used and/or analyzed during the current study will be available from the authors upon reasonable request.
Disclaimer/Publisher's Note
The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of EWA Publishing and/or the editor(s). EWA Publishing and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
About volume
Volume title: Proceedings of CONF-MLA 2024 Workshop: Semantic Communication Based Complexity Scalable Image Transmission System for Resource Constrained Devices
© 2024 by the author(s). Licensee EWA Publishing, Oxford, UK. This article is an open access article distributed under the terms and
conditions of the Creative Commons Attribution (CC BY) license. Authors who
publish this series agree to the following terms:
1. Authors retain copyright and grant the series right of first publication with the work simultaneously licensed under a Creative Commons
Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this
series.
2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the series's published
version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial
publication in this series.
3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and
during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See
Open access policy for details).