Download pdf
Overfitting of CNN model in cifar-10: Problem and solutions
- 1 New York University Shanghai
* Author to whom correspondence should be addressed.
Abstract
CNN, proposed by Yann LeCun in the 1980s, has gained high attention and extensive research from both the academia and the industry. It has proved to be useful in a wide variety of fields, including image recognition, which aims to enable the computer to identify different objects in digital images. Despite its usefulness, problems like overfitting occur during the training process of a CNN model, which seriously harm the effectiveness of the model. Firstly, an initial CNN model is built to accomplish image recognition based on data from cifar-10. Secondly, after the presence of overfitting during the training and validation of the initial model, 4 methods, including shallower model, L2 regularization, dropout, data augmentation, are proposed to see how they handle overfitting respectively, and comparisons are made between different methods. Thirdly, the last three methods are combined to see how they handle overfitting together. Finally, conclusion and possible future work are presented. As it turns out, L2 regularization, dropout and data augmentation all reduce overfitting with some slight differences, but shallower model and the combined method cause underfitting rather than overfitting.
Keywords
Overfitting, Cifar-10, Image Recognition, Shallower Model, L2 Regularization, Dropout, Data Augmentation
[1]. Ying X 2019 Journal of physics: Conference series vol 1168 p 1
[2]. Bejani M M and Mehdi G 2021 A systematic review on overfitting control in shallow and deep neural networks. Artificial Intelligence Review. 54(2021) 1-48
[3]. Li Z, Konstantinos K and Ben G 2020 IEEE transactions on medical imaging vol 40 p 1065-1077
[4]. Santos C F G D and João P P 2022 Avoiding overfitting: A survey on regularization methods for convolutional neural networks. ACM Computing Surveys (CSUR). 54.10s (2022) 1-25
[5]. Rezaeezade A and Lejla B 2022 Regularizers to the Rescue: Fighting Overfitting in DeepLearning-based Side-channel Analysis
[6]. Gupta S et al. 2017 Data Science and Analytics: 4th International Conference on Recent Developments in Science, Engineering and Technology vol 799 p 363-371
[7]. Srivastava N et al. 2014 Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research. 15.1 (2014)1929-1958
[8]. Qian L et al. 2020 IEEE Access vol 8 p 62830-62840
[9]. Shorten C and Taghi M K 2019 A survey on image data augmentation for deep learning. Journal of big data 6.1 (2019)1-48
[10]. Perez L and Jason W 2017 The effectiveness of data augmentation in image classification using deep learning. arXiv preprint arXiv:1712.04621 (2017)
[11]. Zhang H, Lin Z, and Yuan J 2019 11th international conference on wireless communications and signal processing (WCSP) p 1-6
[12]. Phaisangittisagul E 2016 7th International Conference on Intelligent Systems, Modelling and Simulation (ISMS) p 174-179
[13]. Quinto B 2020 Introduction to Machine Learning. Next-Generation Machine Learning with Spark: Covers XGBoost, LightGBM, Spark NLP, Distributed Deep Learning with Keras, and More. 1-27
Cite this article
Xia,Z. (2024). Overfitting of CNN model in cifar-10: Problem and solutions. Applied and Computational Engineering,37,212-221.
Data availability
The datasets used and/or analyzed during the current study will be available from the authors upon reasonable request.
Disclaimer/Publisher's Note
The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of EWA Publishing and/or the editor(s). EWA Publishing and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
About volume
Volume title: Proceedings of the 2023 International Conference on Machine Learning and Automation
© 2024 by the author(s). Licensee EWA Publishing, Oxford, UK. This article is an open access article distributed under the terms and
conditions of the Creative Commons Attribution (CC BY) license. Authors who
publish this series agree to the following terms:
1. Authors retain copyright and grant the series right of first publication with the work simultaneously licensed under a Creative Commons
Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this
series.
2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the series's published
version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial
publication in this series.
3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and
during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See
Open access policy for details).