1. Introduction

Object Detection Object Detection (OD), abbreviated as OD, plays an indispensable role in various computer vision applications, encompassing fields such as autonomous vehicles, surveillance systems, and robotics. The fundamental objective of Object Detection is to identify target objects within an image and determine both their respective categories and positions. The complexity of this task arises from the diverse appearances, shapes, and orientations of different objects, combined with factors like illumination variations and occlusion during image capture. In recent times, the research community has proposed two main methodologies for Object Detection: two-stage models and one-stage models. The former tend to suffer from intricate model architectures and excessive region proposals, leading to computationally expensive operations. Conversely, one-stage models have garnered more attention due to their simplicity in design and cost optimization. Among these models, YOLO [1] stands as a representative one-stage approach that addresses object detection through regression. By processing the input image only once, YOLO efficiently obtains the positions of target objects, their respective categories, and the corresponding confidence probabilities. This streamlined model design enables YOLO to detect target objects significantly faster than traditional two-stage methods, making it a preferred choice for applications on resource-constrained devices such as mobile phones and IoT devices. Despite the effectiveness of the original YOLO models, researchers have continuously proposed variants of YOLO tailored to specific tasks or scenarios [2-11], such as YOLO v2-v5 [12-14], YOLO-LITE [8], YOLO tiny [3], and others. However, a comprehensive investigation into the lightweight adaptations of YOLO and its variants is still lacking. This gap hinders a thorough understanding of the development of lightweight approaches for the OD task. Moreover, with the increasing prevalence of OD on mobile and IoT devices, grasping the lightweight OD trend through future research will be essential for enhancing detection efficiency and real-world applications.

To address this research gap, this paper presents a systematic study of lightweight adaptations based on the YOLO framework. Initially, we provide the necessary background knowledge about the OD task and introduce the representative YOLO models. Subsequently, we conduct a comprehensive examination of recent YOLO-based lightweight OD works from multiple research dimensions, including approaches, results, and application scenarios. Furthermore, we delve into a detailed discussion of the existing related efforts. Finally, based on our study, we analyze potential future research directions to further advance the field of lightweight OD.

2. Object detection

Object detection is a classic task in the field of computer vision, encompassing four fundamental problem categories:

(1) Classification problem: determining the category to which an object in the picture (or a certain area) belongs.

(2) Localization problem: identifying where the target object may appear in the image.

(3) Size problem: ascertaining the size of the target object.

(4) Shape problem: recognizing the shape presented by the target object.

To address these problems, two-stage methods were proposed earlier, with representative methods such as R-CNN, SPP-Net, Fast R-CNN, Faster R-CNN, and R-FCN. The fundamental design concept underlying two-stage object detection is as follows: Firstly, the model generates region proposals (RPs) in the original image, which are pre-selection boxes that may potentially contain objects requiring inspection. Subsequently, the objects undergo classification through a CNN or other advanced learning models. The two-stage methods offer advantages in terms of detection precision, owing to their utilization of an anchor mechanism to extract RPs, thereby enhancing detection performance. However, these methods face limitations concerning detection efficiency and training time costs since each stage necessitates learning prediction results based on data inputs derived from the preceding stage. Additionally, the two-stage approaches are also constrained by false positive rates, as an excess of RPs will inevitably lead to redundant detection outcomes. Consequently, achieving real-time performance with these methods requires high-end hardware and renders them unsuitable for deployment on low-power devices, thus limiting their widespread adoption in real-world scenarios.

In stark contrast to the two-stage methods, one-stage methods execute the classification and localization of target objects through direct deep neural network (DNN) based feature extraction. In comparison to their two-stage counterparts, one-stage approaches exhibit superior detection efficiency in both training and detection stages, as they eliminate the time-consuming construction of RPs. One-stage methods generally acquire the ability to learn generalization features of objects, thus mitigating false positives in detection. However, the detection precision of one-stage methods is limited by the open feature extraction networks they utilize. To address this challenge, one-stage methods typically concentrate on constructing deep models. Unlike simple CNNs or their variants, the one-stage network can be divided into different modules to extract various dimensions of image features.

As a representative lightweight one-stage model for object detection, YOLO [1] employs numerous lightweight strategies to achieve enhanced accuracy and efficiency. The details of YOLO and its variants are expounded upon in section 3.3.

3. Study of lightweight object detection

3.1. Global map

To systematically study lightweight works in object detection (OD), we have collected numerous YOLO-based efforts in recent years. Through careful screening, analysis, and summarization, the key information is presented in Table 1, where each work's first author, representative tool, target, approach, result, and scenario are listed.

Table 1. Representative lightweight works based on YOLO.

3.2. Lightweight of YOLO V1-V5

1) YOLO-V1 [1] represents the initial generation of the YOLO algorithm, primarily devised for target detection. Its key feature lies in enhancing detection speed and reducing false alarm rates. Built upon a CNN framework, YOLO-V1 directly predicts bounding boxes for multiple object categories and locations in a single forward pass. The primary advantage of YOLO-V1 is efficiency, yet it is constrained by scale and resolution, leading to relatively low detection accuracy. Moreover, its ability to handle small objects is comparatively poor.

2) YOLO-V2 [12] constitutes an improvement over YOLO-V1, incorporating a series of techniques such as batch normalization, anchor boxes, convolution expansion, and enhanced activation functions for the convolution layer, all of which further elevate YOLO's detection performance. This results in improved detection accuracy and speed, especially concerning small objects. In comparison to V1, YOLO-V2 features a more complex network structure and an increased number of parameters, leading to lengthier training and inference times.

3) YOLO-V3 [13] introduces more improvements and optimizations, such as multi-scale prediction, feature pyramid network, and upsampling, compared to YOLO-V1 and V2. These optimizations result in YOLO-V3 outperforming YOLO-V2 in terms of detection accuracy, speed, and localization capabilities. YOLO-V3 also supports larger objects and multi-scale detection. However, it is more complex than YOLO-V2 and requires more time and computing resources for training and inference.

4) YOLO-V4 [14] incorporates several technologies, including Scaled-YOLO, CSPDarknet53, CSPResNeXt50, YOLO-V4, and YOLO-V4-neck, which lead to significant improvements in speed and performance. Compared to YOLO-V3, YOLO-V4 shows enhancements in detection accuracy, speed, and robustness. However, YOLO-V4 is more complex, and the resource requirements for training and inference are higher.

5) YOLO-V5 adopts the structure of V4 and improves its portability, making it easier to deploy to new hardware and existing applications by reducing the code base and writing common COCO evaluation scripts. YOLO-V5 also supports various data enhancement methods and model center loss training. Compared to YOLO-V4, YOLO-V5 offers better speed, a smaller model size, easier deployment, and detection accuracy close to YOLO-V4.

3.3. Lightweight of variants

In addition to the YOLO models, there are several variants of YOLO. For instance, Light-YOLO [4] is a new lightweight tool developed based on YOLO-V2 that allows selective channel pruning of discarded channels and spatial refinement of modules. Smart-YOLO [5] is a new lightweight algorithm proposed based on YOLO-V3, utilizing reverse bottleneck blocks and deep separable convolutions. YOLO-LITE [8], constructed on the basis of YOLO-V2, is designed to create a smaller, faster, and more efficient model. YOLO-DFD [16] is an improved YOLO-V4 based lightweight detection algorithm developed to address the problem of dog feces in our living environment. Poly-YOLO [17], based on YOLO-V3, removes rewritten labels and inefficient distribution of anchors. However, the lightweight works for YOLO variants are often designed for non-targeted generalized tasks and are not directly applicable in specific scenarios.

3.4. Specific scenarios

There are also works proposed for specific scenarios. For example, Gao et al. [17] address high power consumption and poor performance in autonomous driving of automobiles by using YOLO-V3 support. Li et al. [18] accurately locate pedestrians at night. Additionally, Zaatouri [19] improves real-time traffic light control, while Tian et al. [9] accurately detect apples in orchards with fluctuating illumination and complex backgrounds. These YOLO-based lightweight works for specific scenarios often lack improvement in model construction.

4. Discussion

4.1. Existing efforts

4.1.1. Advantages. The advantages of the current YOLO-based OD lightweight object detection works are evident in the related publications. Firstly, some of the works aim to improve detection efficiency while maintaining an acceptable level of accuracy. They achieve this goal through improved complex neural network architectures and specific data strategies. This trend can be observed in the evolution of YOLO models. For example, YOLO-V1 predicted the four vertices of the target object using multiple CNN layers and FC, while YOLO-V2 changed the prediction architecture to focus on anchors and grids and increased the prediction categories significantly. YOLO-V3 introduced more complexities in head categories and detection grids. YOLO-V4 further improved the detection head and changed the loss function to CIoU loss. YOLO-V5 also saw corresponding improvements in the backbone network.

4.1.2. Limitations. Our study reveals that YOLO Lightweight is a viable alternative to existing lightweight object detection models. However, there are limitations within the proposed YOLO-based models or applications. Although many existing efforts aim to balance performance in terms of both accuracy and efficiency, the improvements after YOLO-V4 are still limited. Some methods focus more on improving detection accuracy, but the efficiency for popularization and generalization is not satisfactory. Additionally, with the prevalence of applications on the Internet of Things and mobile terminals, the real-time requirements for detection algorithms are increasing. The current optimization models, including YOLO and its variants, are challenging to be directly used in these real-time scenarios. Special pruning or quantization processing is still required to achieve terminal-related adaptation. Furthermore, although many proposed models claim to achieve both high accuracy and efficiency, they are primarily trained on commonly used datasets like PASCAL VOC and COCO. The effectiveness of these models has not been sufficiently verified under new domains, limiting their practicality.

4.2. Future directions

Based on the analysis of both advantages and limitations of existing lightweight efforts, future works in this direction can be divided into the following three points.

4.2.1. Trade-off between accuracy and efficiency. The trade-off between accuracy and efficiency remains a key focus in lightweight object detection. However, most of the existing works excel in one aspect or show an average performance in both aspects, making it challenging to achieve optimal performance in both areas. Future research should focus on constructing new architectures and algorithms that can strike a better balance between accuracy and efficiency. Advanced techniques in computer vision, such as transformers and data augmentation, should also be integrated into the field. Additionally, breakthroughs in lightweight object detection should be combined with technologies in other related fields, such as hardware device transmission and edge device computing speed, involving cloud computing, edge computing, and distribution system.

4.2.2. Exploration of new application scenarios. Object detection, as a fundamental task in computer vision, has diverse applications. The emergence of YOLO-based lightweight models has extended the scope of potential applications. While some detection scenarios, such as agriculture, traffic, and ships, have been explored, more applications are expected to be discovered. This future direction will make lightweight object detection more valuable and meaningful.

4.2.3. Customized lightweight. As seen in the previous analysis, lightweight object detection tends to be customized and specific. Since achieving high accuracy and efficiency simultaneously is challenging, the task can be separated into several attributes based on user requirements. Detection targets can then be transformed into one or more attributes, such as precision, recall, explanation, generalization, etc. This approach allows for the exploration of more practical functionalities of YOLO-based models for users. This promising future direction aims to solve real object detection problems in various industries while considering resource costs.

5. Conclusion

In this paper, we have made the first effort to understand the development of YOLO-based object detection models from the perspective of lightweight. We comprehensively studied and discussed more than twenty YOLO-based lightweight works. Through this study, we have highlighted the development trend for object detection and the YOLO models. We have also analyzed some interesting findings discovered during the investigation of lightweight YOLO models. We believe that this study and its findings can uncover the potential philosophy behind model lightweight and further enhance the applicability of object detection in real-world resource-critical scenarios.