A review of the application of CNN-based computer vision in auto-driving

Research Article
Open access

A review of the application of CNN-based computer vision in auto-driving

Tongwei Zhang 1*
  • 1 Department of Computer Science and Software Engineering (CSSE), Concordia University, Montreal, QC, H3H 2L9, Canada    
  • *corresponding author 330aihebe@gmail.com
ACE Vol.5
ISSN (Print): 2755-273X
ISSN (Online): 2755-2721
ISBN (Print): 978-1-915371-57-7
ISBN (Online): 978-1-915371-58-4

Abstract

Beginning with Tesla, self-driving technology has become commercially available in recent decades. Target recognition and semantic segmentation remain significant obstacles for autonomous driving systems. Given that these two tasks are also part of the primary tasks of computer vision and that deep learning techniques based on convolutional neural networks have made advancements in the field of computer vision, a great deal of research has begun to apply convolutional neural networks to autonomous driving in the past few years. In this paper, we examine recent publications on CNN-based techniques for autonomous driving, classify them, and offer insights into future research directions.

Keywords:

Convolutional Neural Networks, Computer Vision, Autonomous Driving, Image Recognization, Object Detection, Semantic Segmentation

Zhang,T. (2023). A review of the application of CNN-based computer vision in auto-driving. Applied and Computational Engineering,5,69-74.
Export citation

References

[1]. W. Xu, B. Li, S. Liu, and W. Qiu, “Real-time object detection and semantic segmentation for autonomous driving,” Feb. 2018, p. 44. doi: 10.1117/12.2288713.

[2]. A. Voulodimos, N. Doulamis, A. Doulamis, and E. Protopapadakis, “Deep Learning for Computer Vision: A Brief Review,” Computational Intelligence and Neuroscience, vol. 2018, pp. 1–13, 2018, doi: 10.1155/2018/7068349.

[3]. S. Zhou and W. Song, “Deep learning-based roadway crack classification using laser-scanned range images: A comparative study on hyperparameter selection,” Automation in Construction, vol. 114, p. 103171, Jun. 2020, doi: 10.1016/j.autcon.2020.103171.

[4]. J. C. P. Cheng and M. Wang, “Automated detection of sewer pipe defects in closed-circuit television images using deep learning techniques,” Autom. Constr., vol. 95, pp. 155–171, Nov. 2018, doi: 10.1016/j.autcon.2018.08.006.

[5]. M. Wang and J. C. P. Cheng, “A unified convolutional neural network integrated with conditional random field for pipe defect segmentation,” Computer-Aided Civil and Infrastructure Engineering, Jul. 2019, doi: 10.1111/mice.12481.

[6]. M. Dildar et al., “Skin Cancer Detection: A Review Using Deep Learning Techniques,” Int. J. Environ. Res. Public. Health, vol. 18, no. 10, p. 5479, May 2021, doi: 10.3390/ijerph18105479.

[7]. M. ur Rehman, S. H. Khan, S. M. Danish Rizvi, Z. Abbas, and A. Zafar, “Classification of Skin Lesion by Interference of Segmentation and Convolotion Neural Network,” in 2018 2nd International Conference on Engineering Innovation (ICEI), Jul. 2018, pp. 81–85. doi: 10.1109/ICEI18.2018.8448814.

[8]. L. Chen, W. Zhan, W. Tian, Y. He, and Q. Zou, “Deep Integration: A Multi-Label Architecture for Road Scene Recognition,” IEEE Trans. Image Process., vol. 28, no. 10, pp. 4883–4898, 2019, doi: 10.1109/TIP.2019.2913079.

[9]. G. Li et al., “ML-ANet: A Transfer Learning Approach Using Adaptation Network for Multi-label Image Classification in Autonomous Driving,” Chin. J. Mech. Eng. Ji Xie Gong Cheng Xue Bao Engl. Ed, vol. 34, no. 1, Dec. 2021, doi: 10.1186/s10033-021-00598-9.

[10]. G. Li, Y. Yang, and X. Qu, “Deep Learning Approaches on Pedestrian Detection in Hazy Weather,” IEEE Trans. Ind. Electron., vol. 67, no. 10, pp. 8889–8899, 2020, doi: 10.1109/TIE.2019.2945295.

[11]. J. Redmon and A. Farhadi, “YOLO9000: Better, Faster, Stronger,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jul. 2017, pp. 6517–6525. doi: 10.1109/CVPR.2017.690.

[12]. M. Hnewa and H. Radha, “Object Detection Under Rainy Conditions for Autonomous Vehicles: A Review of State-of-the-Art and Emerging Techniques,” IEEE Signal Process. Mag., vol. 38, no. 1, pp. 53–67, 2021, doi: 10.1109/MSP.2020.2984801.

[13]. M.-Y. Liu, T. Breuel, and J. Kautz, “Unsupervised Image-to-Image Translation Networks.” arXiv, Jul. 22, 2018. doi: 10.48550/arXiv.1703.00848.

[14]. Y. Chen, W. Li, C. Sakaridis, D. Dai, and L. Van Gool, “Domain Adaptive Faster R-CNN for Object Detection in the Wild,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 2018, pp. 3339–3348. doi: 10.1109/CVPR.2018.00352.

[15]. N. A. M. Mai, P. Duthon, L. Khoudour, A. Crouzil, and S. A. Velastin, “Sparse LiDAR and Stereo Fusion (SLS-Fusion) for Depth Estimationand 3D Object Detection.” arXiv, May 28, 2021. doi: 10.48550/arXiv.2103.03977.

[16]. G. Li, Y. Yang, X. Qu, D. Cao, and K. Li, “A deep learning based image enhancement approach for autonomous driving at night,” Knowledge-Based Systems, vol. 213, p. 106617, Feb. 2021, doi: 10.1016/j.knosys.2020.106617.

[17]. X. Cheng, P. Wang, and R. Yang, “Learning Depth with Convolutional Spatial Propagation Network,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 42, no. 10, pp. 2361–2379, 2020, doi: 10.1109/TPAMI.2019.2947374.

[18]. X. Ma, Z. Wang, H. Li, P. Zhang, W. Ouyang, and X. Fan, “Accurate Monocular 3D Object Detection via Color-Embedded 3D Reconstruction for Autonomous Driving,” in 2019 IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 6850–6859. doi: 10.1109/ICCV.2019.00695.

[19]. P. Radecki, M. Campbell, and K. Matzen, “All Weather Perception: Joint Data Association, Tracking, and Classification for Autonomous Ground Vehicles.” arXiv, May 07, 2016. doi: 10.48550/arXiv.1605.02196.

[20]. H. Gao, B. Cheng, J. Wang, K. Li, J. Zhao, and D. Li, “Object Classification Using CNN-Based Fusion of Vision and LIDAR in Autonomous Vehicle Environment,” IEEE Trans. Ind. Inform., vol. 14, no. 9, pp. 4224–4231, Sep. 2018, doi: 10.1109/TII.2018.2822828.

[21]. W. Boyuan and W. Muqing, “Study on Pedestrian Detection Based on an Improved YOLOv4 Algorithm,” in 2020 IEEE 6th International Conference on Computer and Communications (ICCC), 2020, pp. 1198–1202. doi: 10.1109/ICCC51575.2020.9344983.

[22]. A. Hechri and A. Mtibba, “Two-Stage Traffic Sign Detection and Recognition Based on SVM and Convolutional Neural Networks,” IET Image Processing, Dec. 2019, doi: 10.1049/iet-ipr.2019.0634.

[23]. J. Müller and K. Dietmayer, "Detecting Traffic Lights by Single Shot Detection," 2018 21st International Conference on Intelligent Transportation Systems (ITSC), 2018, pp. 266-273, doi: 10.1109/ITSC.2018.8569683.

[24]. X.-Y. Ye, D.-S. Hong, H.-H. Chen, P.-Y. Hsiao, and L.-C. Fu, “A two-stage real-time YOLOv2-based road marking detector with lightweight spatial transformation-invariant classification,” Image Vis. Comput., vol. 102, p. 103978, Oct. 2020, doi: 10.1016/j.imavis.2020.103978.

[25]. S. Papadopoulos, I. Mademlis and I. Pitas, "Neural vision-based semantic 3D world modeling," 2021 IEEE Winter Conference on Applications of Computer Vision Workshops (WACVW), 2021, pp. 181-190, doi: 10.1109/WACVW52041.2021.00024.

[26]. A. Kherraki, M. Maqbool, and R. El Ouazzani, “Traffic Scene Semantic Segmentation by Using Several Deep Convolutional Neural Networks,” in 2021 3rd IEEE Middle East and North Africa COMMunications Conference (MENACOMM), 2021, pp. 1–6. doi: 10.1109/MENACOMM50742.2021.9678270.

[27]. S. Papadopoulos, I. Mademlis and I. Pitas, "Semantic Image Segmentation Guided By Scene Geometry," 2021 IEEE International Conference on Autonomous Systems (ICAS), 2021, pp. 1-5, doi: 10.1109/ICAS49788.2021.9551117.

[28]. M. S. S. Mahecha, O. J. S. Parra, and J. B. Velandia, “Design of a System for Melanoma Detection Through the Processing of Clinical Images Using Artificial Neural Networks,” Lecture Notes in Computer Science, pp. 605–616, 2018, doi: 10.1007/978-3-030-02131-3_53.


Cite this article

Zhang,T. (2023). A review of the application of CNN-based computer vision in auto-driving. Applied and Computational Engineering,5,69-74.

Data availability

The datasets used and/or analyzed during the current study will be available from the authors upon reasonable request.

Disclaimer/Publisher's Note

The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of EWA Publishing and/or the editor(s). EWA Publishing and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

About volume

Volume title: Proceedings of the 3rd International Conference on Signal Processing and Machine Learning

ISBN:978-1-915371-57-7(Print) / 978-1-915371-58-4(Online)
Editor:Omer Burak Istanbullu
Conference website: http://www.confspml.org
Conference date: 25 February 2023
Series: Applied and Computational Engineering
Volume number: Vol.5
ISSN:2755-2721(Print) / 2755-273X(Online)

© 2024 by the author(s). Licensee EWA Publishing, Oxford, UK. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license. Authors who publish this series agree to the following terms:
1. Authors retain copyright and grant the series right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this series.
2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the series's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this series.
3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See Open access policy for details).

References

[1]. W. Xu, B. Li, S. Liu, and W. Qiu, “Real-time object detection and semantic segmentation for autonomous driving,” Feb. 2018, p. 44. doi: 10.1117/12.2288713.

[2]. A. Voulodimos, N. Doulamis, A. Doulamis, and E. Protopapadakis, “Deep Learning for Computer Vision: A Brief Review,” Computational Intelligence and Neuroscience, vol. 2018, pp. 1–13, 2018, doi: 10.1155/2018/7068349.

[3]. S. Zhou and W. Song, “Deep learning-based roadway crack classification using laser-scanned range images: A comparative study on hyperparameter selection,” Automation in Construction, vol. 114, p. 103171, Jun. 2020, doi: 10.1016/j.autcon.2020.103171.

[4]. J. C. P. Cheng and M. Wang, “Automated detection of sewer pipe defects in closed-circuit television images using deep learning techniques,” Autom. Constr., vol. 95, pp. 155–171, Nov. 2018, doi: 10.1016/j.autcon.2018.08.006.

[5]. M. Wang and J. C. P. Cheng, “A unified convolutional neural network integrated with conditional random field for pipe defect segmentation,” Computer-Aided Civil and Infrastructure Engineering, Jul. 2019, doi: 10.1111/mice.12481.

[6]. M. Dildar et al., “Skin Cancer Detection: A Review Using Deep Learning Techniques,” Int. J. Environ. Res. Public. Health, vol. 18, no. 10, p. 5479, May 2021, doi: 10.3390/ijerph18105479.

[7]. M. ur Rehman, S. H. Khan, S. M. Danish Rizvi, Z. Abbas, and A. Zafar, “Classification of Skin Lesion by Interference of Segmentation and Convolotion Neural Network,” in 2018 2nd International Conference on Engineering Innovation (ICEI), Jul. 2018, pp. 81–85. doi: 10.1109/ICEI18.2018.8448814.

[8]. L. Chen, W. Zhan, W. Tian, Y. He, and Q. Zou, “Deep Integration: A Multi-Label Architecture for Road Scene Recognition,” IEEE Trans. Image Process., vol. 28, no. 10, pp. 4883–4898, 2019, doi: 10.1109/TIP.2019.2913079.

[9]. G. Li et al., “ML-ANet: A Transfer Learning Approach Using Adaptation Network for Multi-label Image Classification in Autonomous Driving,” Chin. J. Mech. Eng. Ji Xie Gong Cheng Xue Bao Engl. Ed, vol. 34, no. 1, Dec. 2021, doi: 10.1186/s10033-021-00598-9.

[10]. G. Li, Y. Yang, and X. Qu, “Deep Learning Approaches on Pedestrian Detection in Hazy Weather,” IEEE Trans. Ind. Electron., vol. 67, no. 10, pp. 8889–8899, 2020, doi: 10.1109/TIE.2019.2945295.

[11]. J. Redmon and A. Farhadi, “YOLO9000: Better, Faster, Stronger,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jul. 2017, pp. 6517–6525. doi: 10.1109/CVPR.2017.690.

[12]. M. Hnewa and H. Radha, “Object Detection Under Rainy Conditions for Autonomous Vehicles: A Review of State-of-the-Art and Emerging Techniques,” IEEE Signal Process. Mag., vol. 38, no. 1, pp. 53–67, 2021, doi: 10.1109/MSP.2020.2984801.

[13]. M.-Y. Liu, T. Breuel, and J. Kautz, “Unsupervised Image-to-Image Translation Networks.” arXiv, Jul. 22, 2018. doi: 10.48550/arXiv.1703.00848.

[14]. Y. Chen, W. Li, C. Sakaridis, D. Dai, and L. Van Gool, “Domain Adaptive Faster R-CNN for Object Detection in the Wild,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 2018, pp. 3339–3348. doi: 10.1109/CVPR.2018.00352.

[15]. N. A. M. Mai, P. Duthon, L. Khoudour, A. Crouzil, and S. A. Velastin, “Sparse LiDAR and Stereo Fusion (SLS-Fusion) for Depth Estimationand 3D Object Detection.” arXiv, May 28, 2021. doi: 10.48550/arXiv.2103.03977.

[16]. G. Li, Y. Yang, X. Qu, D. Cao, and K. Li, “A deep learning based image enhancement approach for autonomous driving at night,” Knowledge-Based Systems, vol. 213, p. 106617, Feb. 2021, doi: 10.1016/j.knosys.2020.106617.

[17]. X. Cheng, P. Wang, and R. Yang, “Learning Depth with Convolutional Spatial Propagation Network,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 42, no. 10, pp. 2361–2379, 2020, doi: 10.1109/TPAMI.2019.2947374.

[18]. X. Ma, Z. Wang, H. Li, P. Zhang, W. Ouyang, and X. Fan, “Accurate Monocular 3D Object Detection via Color-Embedded 3D Reconstruction for Autonomous Driving,” in 2019 IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 6850–6859. doi: 10.1109/ICCV.2019.00695.

[19]. P. Radecki, M. Campbell, and K. Matzen, “All Weather Perception: Joint Data Association, Tracking, and Classification for Autonomous Ground Vehicles.” arXiv, May 07, 2016. doi: 10.48550/arXiv.1605.02196.

[20]. H. Gao, B. Cheng, J. Wang, K. Li, J. Zhao, and D. Li, “Object Classification Using CNN-Based Fusion of Vision and LIDAR in Autonomous Vehicle Environment,” IEEE Trans. Ind. Inform., vol. 14, no. 9, pp. 4224–4231, Sep. 2018, doi: 10.1109/TII.2018.2822828.

[21]. W. Boyuan and W. Muqing, “Study on Pedestrian Detection Based on an Improved YOLOv4 Algorithm,” in 2020 IEEE 6th International Conference on Computer and Communications (ICCC), 2020, pp. 1198–1202. doi: 10.1109/ICCC51575.2020.9344983.

[22]. A. Hechri and A. Mtibba, “Two-Stage Traffic Sign Detection and Recognition Based on SVM and Convolutional Neural Networks,” IET Image Processing, Dec. 2019, doi: 10.1049/iet-ipr.2019.0634.

[23]. J. Müller and K. Dietmayer, "Detecting Traffic Lights by Single Shot Detection," 2018 21st International Conference on Intelligent Transportation Systems (ITSC), 2018, pp. 266-273, doi: 10.1109/ITSC.2018.8569683.

[24]. X.-Y. Ye, D.-S. Hong, H.-H. Chen, P.-Y. Hsiao, and L.-C. Fu, “A two-stage real-time YOLOv2-based road marking detector with lightweight spatial transformation-invariant classification,” Image Vis. Comput., vol. 102, p. 103978, Oct. 2020, doi: 10.1016/j.imavis.2020.103978.

[25]. S. Papadopoulos, I. Mademlis and I. Pitas, "Neural vision-based semantic 3D world modeling," 2021 IEEE Winter Conference on Applications of Computer Vision Workshops (WACVW), 2021, pp. 181-190, doi: 10.1109/WACVW52041.2021.00024.

[26]. A. Kherraki, M. Maqbool, and R. El Ouazzani, “Traffic Scene Semantic Segmentation by Using Several Deep Convolutional Neural Networks,” in 2021 3rd IEEE Middle East and North Africa COMMunications Conference (MENACOMM), 2021, pp. 1–6. doi: 10.1109/MENACOMM50742.2021.9678270.

[27]. S. Papadopoulos, I. Mademlis and I. Pitas, "Semantic Image Segmentation Guided By Scene Geometry," 2021 IEEE International Conference on Autonomous Systems (ICAS), 2021, pp. 1-5, doi: 10.1109/ICAS49788.2021.9551117.

[28]. M. S. S. Mahecha, O. J. S. Parra, and J. B. Velandia, “Design of a System for Melanoma Detection Through the Processing of Clinical Images Using Artificial Neural Networks,” Lecture Notes in Computer Science, pp. 605–616, 2018, doi: 10.1007/978-3-030-02131-3_53.