1. Introduction
People prefer air transportation over large distances. People travelling with baggage should handover them to the cargo holdings. There are certain restrictions that a baggage should be within the required size and weight. Baggage’s are often checked at airline systems to detect the threat items such as knife, gun, blade, and explosive devices etc., The baggage is transferred through the panel to check x-ray screening. The panel recognizes the threat items through imaging. The service agents will check the threat presence in the baggage through imaging from the panel and print the details of the passenger on a tag which will be notified as checked baggage. The research papers in the references had brought a new state-of-art methodologies in detecting the threat items in the baggage screening through Deep Learning (DL). The detection may be a masked image or may be the confidence percent in bounding box representation. The researchers use some custom datasets like either X-ray images or videos and custom models to build the state-of-art models.
2. Literature survey
There are many existing works for detecting the threat items and classifying the information with various datasets. In this survey, the state-of-art models of much research will be discussed. Taimur Hassan et al., proposed an unsupervised anomaly localization. They made a one-staged encoder-decoder utilization network to train the dataset at a time. The further testing was made, and they achieved mean AP for five publicly available datasets individually. They scored 0.85 on GDXray which performs best on their proposed model [1]. The researchers presented a Hierarchical CNN model to the custom dataset which is named as ASS-BD which have video clips as datasets. The clips were taken at the airport baggage checking recording scenes. At last, they succeeded in increasing their mean AP for their custom dataset [2]. The author Wei Zhang et al., introduced a detection model through an automated mode for threat items based on depth wise and combination of three pretrained models. One of the best parts of this model is high detection accuracy, fast computational speed [3]. Priscilla Steno et al., made research which aims to the improvement of faster R-CNN. The researchers made an improved regional based proposal of faster R-CNN. Therefore, the proposed model shown a 15% of improvement to their existing method on localizing the threat items [4]. The researchers made a frequency resolution increase on the input images that are being used. They have employed a multi-scale convolution neural network (CNN) for identifying the material class based on the Dual Energy X-ray scans (DEXS) data which is limited to six fixed classes [5]. Hyo-young et al., proposed learning-based image synthesis for hazardous object detection. They have use average precision (AP) and mean AP for object detection performance. Finally, the proposed method improves performance of various object detection [6]. Xianglong Liu et al proposed an over-sampling de-occlusion attention network (DOAM-O).This model improves the rate of performance of some famous object detection models like SSD, YOLOv3. The models also achieved an impressive result [7]. The researchers used DL based models and CV algorithms to combine large number of x-ray images. They have categorized the threat items as knives, blade, gun, and shuriken. After the evaluation of each category, they have obtained mean AP precisely [8]. This paper tackles the problem through the pixel-based approach. Since the pixels in X-Ray images shows the radiation energy and thus signifies that the darker or denser the pixel regions are the higher the chance of being overlapped objects. The authors also noted that Mask RCNN method is more accurate (93%) than Faster RCNN where the most items exhibit dense material properties [9]. The authors Boying Wang et al proposed Real-world Prohibited Item Detection. The datasets they have used for this work are PIDray dataset. They have use AP, AR, and IOU metrics. Finally, they show that this model will help to evaluate the prohibited items recognition in real applications [10]. The authors Taimur Hassan et al presented a single-staged instance segmentation framework and it is only the framework that has been validated on combined grayscale and colored scans. Mean AP obtained by using the proposed method on GDXray is 96.72 [11]. The authors of this paper proposed a novel detection strategy called as Cascaded Structure Tensor (CST) is to identify obstructed or overlapped objects. This scheme works on any grayscale or colored X-ray scan for recognizing the potential threats [12]. The authors hyo-young Kim et al., proposed detail restoration and tone mapping networks. Generally, there is no publicly available dataset we have constructed a synthetic dataset. Finally, they show that proposed method achieves visually compelling images with improved details [13]. The authors Bangzhong Gu et al., proposed Automatic and robust object detection in X-ray baggage inspection. The datasets they have used for this work are Xdb1 and Xdb2 datasets. The automation work achieved as best when compared with existing models [14]. Neelanjan Bhowmik et al made a Transfer Learning which focuses on the automated screening approaches. They have considered 2D x-ray images to localize the threat items. They have used automated faster R-CNN to be classifying and determining. The results produce better accuracy [15]. Qing Ma et al proposed a DL model which is named as Unfolding Spatiospectral Super-Resolution Network (US3RN). They stated that the model reduces the parameters to extremely small. They have named this method as two-step method. The achieved PSNR is 32.90 and SSIM is 0.9416 [16].
Sonali et al proposed a technique which filters the noisy area and low contrast areas and creates sharpened images that highlight the object area in an image. The proposed model is known as CLAHE. The model exceeded PSNR improvement by 7.85% and SSIM improvement by 1.19% [17].
2.1. Hierarchical diagram
The hierarchical diagram is a representation made from the related works.
|
Figure 1. A Hierarchical view of technique, Datasets, and their respective evaluations. |
The figure 1 represents the techniques of the corresponding references and stated as numbers. The datasets which are used in all the references are also stated as numbers. The evaluations of the techniques through the datasets in the figure are classified as mAP, Det AP, FSITM, and Accuracy.
3. Analysis on existing methodologies
The analysis is made in a tabular representation, consisting of author’ names, technical details, advantages of their proposed work, limitations in their work, and gaps or future scope.
3.1. Comparison table
Table 1 (a). Comparison table of related works which consists of techniques, advantages, limitations, and gaps. | ||||||
Year | Technique | Advantages | Limitations | Results | Gaps | |
1 | 2021 | Encoder-decoder | It is one-staged unsupervised approach which consists encoder-decoder network. This approach reduces time and gives best. | The working of SIXray dataset is the largest and most challenging. The model trained without supervision. | mAP=85.9% | Overlapping object detection. |
2 | 2021 | Hierarchical CNN | The model has strong versatility when applied inspection task of appearance. It also solves motion blur, and deformation. | Difficult frames for detecting such as part occlusion, rare pose when the operator is placing baggage, tying baggage tags. | mAP=74.8% | JETSON TX2 is used for efficient computing. |
3 | 2021 | ThreNet | The model has faster computation speed and bounding boxes can cover threat object perfectly. | The detection depends on some specific region rather than the entire image. The model loss some information in detection. | mAP=96.1% | Anchor-free detection |
4 | 2021 | Enhanced Faster R-CNN | The enhance faster R-CNN with improved region proposal network. Improved RPN produces better quality feature map. | The proposed model is only trained to detect the pistols in the screening. It cannot detect other objects. | Accuracy=77% | Train model with larger dataset |
Table 1 (b).
Year | Technique | Advantages | Limitations | Results | Gaps | ||
5 | 2021 | Multi CNN | Less training time, less parameters and a multi scaled model which is useful for different level feature extraction. | It is limited for classification of six fixed classes of object materials and prone to noise. | Accuracy=95% Recall=96% | Work on noisy images | |
6 | 2021 | Learning -based image synthesis | The model shows the improvement in performance of various objects compared to previous image synthesis. | The model is a conventional method and failed in detecting occluded objects, although cause false alarms by test set. | mAP=77.1% | More hard training samples to be taken. | |
7 | 2021 | DOAM-O | Effective in collecting edge and region information to improve feature representation in occluded objects. | Extra computational cost compared with existing methods. Difficulty in dealing with high-level occlusions. | mAP=83.8% | Integrated with any other model. | |
8 | 2021 | YOLOv3 | The idea of the work is to show how to use known object detection strategies in baggage inspection, rather than new. | The subset of test set contains same categories but different images from training set. Thus, it is challenging for detection. | mAP=90% | Superimposition of simulated targets | |
9 | 2021 | Faster R-CNN | High prediction confidence on true positives and consistent on false positives. Supress any potentially wrong detections. | When most of the objects are overlapped then this model has a time detecting since the baggage has more denser materials. | mAP=91.4% | Need to improve of finding wrench and pliers | |
10 | 2021 | Selective dense attention network | SDANet achieves best performance in terms of all subsets in PIDray dataset. | Detecting prohibited items hidden in messy objects is one of the most significant challenges in security inspection. | Det AP=48.9% | Include more images in dataset. | |
11 | 2020 | Trainable structure tensor | Ability to suppress the boundaries of normal baggage content as it is tested on multiple datasets and robust in detecting. | The proposed framework is leveraged on pixel-wise recognition, some pixels are classified as false negatives. | mAP=95.1% | Extended for 3D printed objects | |
12 | 2020 | Learning-driven tensor-shot | Works on any grey scale or coloured scan for recognising threat objects. The model can detect even for overlapped objects. | Misses some extremely dull and occluded objects. Inability of the model to accurately differentiate the low contrasted objects within the low-energy tensor. | mAP=91.62% F1-score=95% | Backbone should be more biased. | |
13 | 2020 | TM-Net and DR-Net | Tone Mapping is simple, fast and preserve intensity. Finally, they show that proposed method achieves visually compelling images with improved details. | There is no publicly available dataset for TM of X-ray images. we constructed synthetic dataset to simulate X-ray images for target applications. | FSITM=93% | Further improve performance in terms of visibility of images. | |
14 | 2020 | Mask R-CNN | The proposed method proves that it is more accurate and robust when dealing with densely cluttered backgrounds during X-ray baggage inspection. | X-ray object detection is still a challenge for them with characteristics and densely cluttered background. | mAP=95.4% | Discriminating the false alarms. | |
15 | 2019 | Automated faster R-CNN | Here they present a deep learning enabled approach to address the challenges of threat concealment detection within consumer electronics items via a generalized approach to anomaly, rather than explicit threat, detection | The dataset consists of disparate models and shapes of electronics items, where the inserted anomaly/threat concealment might be challenging to identify. | Accuracy=97% Precision=95% | Targeting on more varied electronic items | |
Table 1 (c).
Year | Technique | Advantages | Limitations | Results | Gaps | ||
16 | 2022 | UnifoldS3RN | The design of the US3RN provides a fast spatial global operation to accelerate the spatial SR network. | The training of NTIRE2020 database made a challenge for them to make spectral reconstruction for RGB images. | PSNR=32.9dB SSIM=94.16% | Extensive ablation study will be made. | |
17 | 2019 | CLAHE | It is proved that the proposed technique removes noise and enhances contrast in fundus images effectively. The results of the model are quite impressive. | The image enhancement for RGB images shown a quite challenging one because the model has filter each of the RGB component and de-noise the region and then merge the RGB components. | PSNR=36.2dB SSIM=96.5% | Applied to another medical images. | |
4. Results
|
Figure 2. Graph for evaluations metrics (SSIM, Det AP, FSITM, Accuracy, mAP). |
The Figure 2 graph shows that the significant improvement on their state-of-art models via some performance metrics such as SSIM, Detection AP, FSITM, Accuracy, and Mean AP.
5. Conclusion
This paper gives the overview of the various of state-of-art models and a detailed comparison table to identify the advantages of their methodologies, limitations of their work, and future works which will be implemented. The analysis of the most of the reference’s states that the detection of threat items is done through mask generation and bounding box prediction.
6. Future Scope
After analysing all the reference papers, some gaps are identified on the main theme i.e., airport security screening through DL [18]. Obviously, the major thing of any research is to improve the rate of performance and stability of their model [19]. To improve the detection rate, we have two theories to propose. These theories based on Computer Vision (CV). We have also analysed the papers 16, 17 for our theories. The dataset which will be trained can be pre-processed [20]. The CV algorithms like Enhance Super Resolution (ESR), and Image Enhancement (IE) can be used to increase the detection rate and bounding box prediction [21]. The dataset will be generated as normal images, ESR images, and IE images. Later, we can use some object detection models to confront our theories with the performance of their results.
References
[1]. Hassan, T., Akçay, S., Bennamoun, M., Khan, S., & Werghi, N. (2021). Unsupervised anomaly instance segmentation for baggage threat recognition. Journal of Ambient Intelligence and Humanized Computing, 1-12.
[2]. Gao, Q., & Liang, P. (2021). Airline baggage appearance transportability detection based on A novel dataset and sequential hierarchical sampling CNN model. IEEE Access, 9, 41833-41843.
[3]. Wei, Y., Zhu, Z., Yu, H., & Zhang, W. (2021). An automated detection model of threat objects for X-ray baggage inspection based on depthwise separable convolution. Journal of Real-Time Image Processing, 18(3), 923-935.
[4]. Steno, P., Alsadoon, A., Prasad, P. W. C., Al-Dala’in, T., & Alsadoon, O. H. (2021). A novel enhanced region proposal network and modified loss function: threat object detection in secure screening using deep learning. The Journal of Supercomputing, 77(4), 3840-3869.
[5]. Benedykciuk, E., Denkowski, M., & Dmitruk, K. (2021). Material classification in X-ray images based on multi-scale CNN. Signal, Image and Video Processing, 15(6), 1285-1293.
[6]. Kim, H. Y., Cho, S. J., Baek, S. J., Jung, S. W., & Ko, S. J. (2021). Learning-Based Image Synthesis for Hazardous Object Detection in X-Ray Security Applications. IEEE Access, 9, 135256-135265.
[7]. Tao, R., Wei, Y., Li, H., Liu, A., Ding, Y., Qin, H., & Liu, X. (2021). Over-sampling de-occlusion attention network for prohibited items detection in noisy X-ray images. arXiv preprint arXiv:2103.00809.
[8]. Saavedra, D., Banerjee, S., & Mery, D. (2021). Detection of threat objects in baggage inspection with X-ray images using deep learning. Neural Computing and Applications, 33(13), 7803-7819.
[9]. Dumagpi, J. K., & Jeong, Y. J. (2021). Pixel-Level Analysis for Enhancing Threat Detection in Large-Scale X-ray Security Images. Applied Sciences, 11(21), 10261.
[10]. Wang, B., Zhang, L., Wen, L., Liu, X., & Wu, Y. (2021). Towards real-world prohibited item detection: A large-scale x-ray benchmark. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 5412-5421).
[11]. Hassan, T., & Werghi, N. (2020). Trainable structure tensors for autonomous baggage threat detection under extreme occlusion. In Proceedings of the Asian Conference on Computer Vision.
[12]. Hassan, T., Shafay, M., Akçay, S., Khan, S., Bennamoun, M., Damiani, E., & Werghi, N. (2020). Meta-transfer learning driven tensor-shot detector for the autonomous localization and recognition of concealed baggage threats. Sensors, 20(22), 6450.
[13]. Kim, H. Y., Park, S., Shin, Y. G., Jung, S. W., & Ko, S. J. (2020). Detail restoration and tone mapping networks for x-ray security inspection. IEEE Access, 8, 197473-197483.
[14]. Gu, B., Ge, R., Chen, Y., Luo, L., & Coatrieux, G. (2020). Automatic and robust object detection in x-ray baggage inspection using deep convolutional neural networks. IEEE Transactions on Industrial Electronics, 68(10), 10248-10257.
[15]. Bhowmik, N., Gaus, Y. F. A., & Breckon, T. P. (2019, November). Using deep neural networks to address the evolving challenges of concealed threat detection within complex electronic items. In 2019 IEEE International Symposium on Technologies for Homeland Security (HST) (pp. 1-6). IEEE.
[16]. Ma, Q., Jiang, J., Liu, X., & Ma, J. (2021). Deep unfolding network for spatiospectral image super-resolution. IEEE Transactions on Computational Imaging, 8, 28-40.
[17]. Sahu, S., Singh, A. K., Ghrera, S. P., & Elhoseny, M. (2019). An approach for de-noising and contrast enhancement of retinal fundus image using CLAHE. Optics & Laser Technology, 110, 87-98.
[18]. Zhang, M., Wang, X., Sathishkumar, V. E., & Sivakumar, V. (2021). Machine learning techniques based on security management in smart cities using robots. Work, 68(3), 891-902.
[19]. Liu, Y., Sathishkumar, V. E., & Manickam, A. (2022). Augmented reality technology based on school physical education training. Computers and Electrical Engineering, 99, 107807.
[20]. Sathishkumar, V. E., Park, J., & Cho, Y. (2020). Using data mining techniques for bike sharing demand prediction in metropolitan city. Computer Communications, 153, 353-366.
[21]. VE, S., Shin, C., & Cho, Y. (2021). Efficient energy consumption prediction model for a data analytic-enabled industry building in a smart city. Building Research & Information, 49(1), 127-143.
Cite this article
Suraj,C.;Malyada,A.;Anjali,B.;Bhagavan,R.;Vardhan,H.;Karrothu,A. (2023). Analysis on airport security of baggage screening using deep learning. Applied and Computational Engineering,4,260-266.
Data availability
The datasets used and/or analyzed during the current study will be available from the authors upon reasonable request.
Disclaimer/Publisher's Note
The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of EWA Publishing and/or the editor(s). EWA Publishing and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
About volume
Volume title: Proceedings of the 3rd International Conference on Signal Processing and Machine Learning
© 2024 by the author(s). Licensee EWA Publishing, Oxford, UK. This article is an open access article distributed under the terms and
conditions of the Creative Commons Attribution (CC BY) license. Authors who
publish this series agree to the following terms:
1. Authors retain copyright and grant the series right of first publication with the work simultaneously licensed under a Creative Commons
Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this
series.
2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the series's published
version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial
publication in this series.
3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and
during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See
Open access policy for details).
References
[1]. Hassan, T., Akçay, S., Bennamoun, M., Khan, S., & Werghi, N. (2021). Unsupervised anomaly instance segmentation for baggage threat recognition. Journal of Ambient Intelligence and Humanized Computing, 1-12.
[2]. Gao, Q., & Liang, P. (2021). Airline baggage appearance transportability detection based on A novel dataset and sequential hierarchical sampling CNN model. IEEE Access, 9, 41833-41843.
[3]. Wei, Y., Zhu, Z., Yu, H., & Zhang, W. (2021). An automated detection model of threat objects for X-ray baggage inspection based on depthwise separable convolution. Journal of Real-Time Image Processing, 18(3), 923-935.
[4]. Steno, P., Alsadoon, A., Prasad, P. W. C., Al-Dala’in, T., & Alsadoon, O. H. (2021). A novel enhanced region proposal network and modified loss function: threat object detection in secure screening using deep learning. The Journal of Supercomputing, 77(4), 3840-3869.
[5]. Benedykciuk, E., Denkowski, M., & Dmitruk, K. (2021). Material classification in X-ray images based on multi-scale CNN. Signal, Image and Video Processing, 15(6), 1285-1293.
[6]. Kim, H. Y., Cho, S. J., Baek, S. J., Jung, S. W., & Ko, S. J. (2021). Learning-Based Image Synthesis for Hazardous Object Detection in X-Ray Security Applications. IEEE Access, 9, 135256-135265.
[7]. Tao, R., Wei, Y., Li, H., Liu, A., Ding, Y., Qin, H., & Liu, X. (2021). Over-sampling de-occlusion attention network for prohibited items detection in noisy X-ray images. arXiv preprint arXiv:2103.00809.
[8]. Saavedra, D., Banerjee, S., & Mery, D. (2021). Detection of threat objects in baggage inspection with X-ray images using deep learning. Neural Computing and Applications, 33(13), 7803-7819.
[9]. Dumagpi, J. K., & Jeong, Y. J. (2021). Pixel-Level Analysis for Enhancing Threat Detection in Large-Scale X-ray Security Images. Applied Sciences, 11(21), 10261.
[10]. Wang, B., Zhang, L., Wen, L., Liu, X., & Wu, Y. (2021). Towards real-world prohibited item detection: A large-scale x-ray benchmark. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 5412-5421).
[11]. Hassan, T., & Werghi, N. (2020). Trainable structure tensors for autonomous baggage threat detection under extreme occlusion. In Proceedings of the Asian Conference on Computer Vision.
[12]. Hassan, T., Shafay, M., Akçay, S., Khan, S., Bennamoun, M., Damiani, E., & Werghi, N. (2020). Meta-transfer learning driven tensor-shot detector for the autonomous localization and recognition of concealed baggage threats. Sensors, 20(22), 6450.
[13]. Kim, H. Y., Park, S., Shin, Y. G., Jung, S. W., & Ko, S. J. (2020). Detail restoration and tone mapping networks for x-ray security inspection. IEEE Access, 8, 197473-197483.
[14]. Gu, B., Ge, R., Chen, Y., Luo, L., & Coatrieux, G. (2020). Automatic and robust object detection in x-ray baggage inspection using deep convolutional neural networks. IEEE Transactions on Industrial Electronics, 68(10), 10248-10257.
[15]. Bhowmik, N., Gaus, Y. F. A., & Breckon, T. P. (2019, November). Using deep neural networks to address the evolving challenges of concealed threat detection within complex electronic items. In 2019 IEEE International Symposium on Technologies for Homeland Security (HST) (pp. 1-6). IEEE.
[16]. Ma, Q., Jiang, J., Liu, X., & Ma, J. (2021). Deep unfolding network for spatiospectral image super-resolution. IEEE Transactions on Computational Imaging, 8, 28-40.
[17]. Sahu, S., Singh, A. K., Ghrera, S. P., & Elhoseny, M. (2019). An approach for de-noising and contrast enhancement of retinal fundus image using CLAHE. Optics & Laser Technology, 110, 87-98.
[18]. Zhang, M., Wang, X., Sathishkumar, V. E., & Sivakumar, V. (2021). Machine learning techniques based on security management in smart cities using robots. Work, 68(3), 891-902.
[19]. Liu, Y., Sathishkumar, V. E., & Manickam, A. (2022). Augmented reality technology based on school physical education training. Computers and Electrical Engineering, 99, 107807.
[20]. Sathishkumar, V. E., Park, J., & Cho, Y. (2020). Using data mining techniques for bike sharing demand prediction in metropolitan city. Computer Communications, 153, 353-366.
[21]. VE, S., Shin, C., & Cho, Y. (2021). Efficient energy consumption prediction model for a data analytic-enabled industry building in a smart city. Building Research & Information, 49(1), 127-143.