1. Introduction

As urban areas expand and economies surge, railways have become a vital mode of transportation for both passenger and freight services. With the increasing density of rail traffic, the heavy maintenance workload for railway equipment has also risen. Consequently, the use of artificial intelligence to replace manual inspection and monitoring tasks is an inevitable trend for the future.In recent years, the rapid development of the drone industry has given rise to drone inspection operations, which have been widely applied in fields such as power, agriculture, rescue, and military. The use of drones equipped with various sensors to inspect railway equipment and operational environments has also become a trend [1]. This paper focuses on the current state of drone visual inspection technology, analyzing the inspection of high-altitude equipment and environments, ground-level equipment and environments, and disaster prevention and control. It particularly reviews the key technical issues in intelligent processing of inspection visual data, discusses its limitations, and explores future developments, providing a reference for related research.

2. Introduction to Drone Vision Technology

The railway drone inspection process mainly includes three stages: pre-inspection preparation, data collection, and data processing. During the pre-inspection preparation stage, an inspection plan and route must be designed based on the inspection objectives, requirements, and environment. Table 1 lists commonly used sensors, including infrared thermal imaging, visible light imaging, and LiDAR.

Table 1. Sensor Classification and Pros and Cons.

Currently, the predominant methods involve the use of single-camera photography, oblique photography, and LiDAR (Light Detection and Ranging). Oblique photography employs multiple cameras to capture images from both vertical and oblique angles [2]. As shown in Figure 1, compared to single-camera photography, oblique photography more accurately reflects the three-dimensional conditions of the terrain. LiDAR utilizes a laser scanning system to collect point cloud data of the test object, which can penetrate vegetation to some extent and obtain more reliable topographical features.

The processing of visual inspection data primarily employs two methods for result analysis: (1) For two-dimensional images, data analysis is conducted using manual or intelligent recognition methods. (2) A three-dimensional reconstruction model is generated, which allows for intuitive observation and measurement evaluation, and effectively preserves models from different periods to aid in subsequent analysis and comparison. Drone image processing technology encompasses image acquisition, image preprocessing, image analysis, and image recognition. Image analysis involves extracting meaningful information from images, such as features, edges, and corners. Image recognition is the process of categorizing images into predefined classes.

3. Aerial Surveillance by Drones

3.1. Inspection of Overhead Contact Systems by Drones

The overhead contact line system (OCLS), a critical component of the railway power supply network, is perpetually exposed to the elements, necessitating meticulous inspection to detect anomalies and potential hazards in fasteners, conductors, insulators, poles, and adjacent right-of-way corridors. Traditionally, these inspections have been carried out predominantly through manual labor or via the 4C/6C systems employed by railway authority electrical departments. However, such methods are constrained by the limitations of operational windows and visual obstructions. Addressing the intricate challenge of fault detection in railway insulators against a cluttered backdrop, Wang Z[4] et al. introduced a detection and classification network rooted in deep convolutional neural networks (CNNs). Leveraging the CSDN framework, they achieved accelerated detection rates without compromising accuracy.

With a focus on the condition assessment of bolts and cotter pins in high-speed railway OCS components captured by UAVs, Yan Xumeng[5] proposed an enhanced EfficientDet model for pinpointing these elements. Subsequently, an improved DeepLabv3+ was employed for instance segmentation and status analysis of bolts and cotter pins. The empirical results revealed that both the false negative rate and the false positive rate were below 8%, demonstrating the efficacy of the methodology.

3.2. Bridge Inspection Operations Utilizing Drones

Given the diverse structural configurations of bridges and their often challenging environments, regular bridge inspections are indispensable for ensuring the safety and reliability of railway operations. Traditional bridge inspection methodologies, involving manual climbing or crane operations, are plagued by inefficiencies, high costs, and significant risks, not to mention the potential for blind spots and subjective human errors. Thus, employing UAVs to chart inspection routes tailored to bridge characteristics and surroundings allows for targeted photography and defect examination [6].

Presently, UAV-based railway bridge inspections encompass bolt loss detection, crack identification in bridge bodies, damage to cable polyethylene protective layers, and displacement monitoring of supports. To address the limitations of conventional railway bridge inspection techniques, Ma Xuezhi [7] et al., proposed a concrete bridge inspection system utilizing UAVs. For cracks typically ranging from 0.2 to 1.0 millimeters in width, the authors employed UAV image acquisition, stitching, and trained a semantic segmentation network based on PSPNet for distress recognition from a sample library of defects.

Zonghan Mu [8] confronted the challenges of large image sizes, drastic variations in target scales, and the small, dense nature of inspection targets when inspecting steel beams on railway bridges with UAV technology. By refining the original YOLOv5s network model, the algorithm was fortified to enhance generalization capabilities on small UAV datasets specific to railway bridge steel structures, thereby elevating detection accuracy.

3.3. Drone-Assisted Rockface Inspection of Mountains and Disaster Assessment at Tunnel Entrances and Exits

The threat posed by unstable rock formations to railway safety is a persistent concern, as rock weathering, erosion, and gravitational forces endanger train operations along the route. Traditional ground-based surveys for hazardous rock masses have been laborious, costly, and fraught with danger, often resulting in incomplete data due to challenging geological conditions. The advent of drone reconnaissance has markedly improved the efficiency and quality of these surveys.

In 2019, Feng Wei [9], taking the Sichuan-Tibet Railway as a case study, employed drones equipped with cameras and airborne radar to explore treacherous high-altitude and cold regions. Tailored to distinct geomorphic features, the investigation combined oblique photogrammetry and radar laser scanning to extract structural plane information and discriminate adverse geological conditions of rock masses.

In 2020, Zhou Fujun [10], using the CZ Railway project as an exemplar, applied drone oblique photogrammetry to reconstruct three-dimensional models of bedrock mountainous areas. The resultant real-world models enabled the identification, measurement, and analysis of rock mass structural planes, enhancing survey efficiency, precision, and safety.

Addressing tunnel entrances and exits, Wang Dong [11] in 2018 utilized drone photogrammetry to measure and assess the stability of steep slopes and hazardous rock masses at the Ji'ermu Tunnel of the Chengdu-Kunming Railway. He proposed a volumetric measurement method for combinations of rock mass structural planes, furnishing critical data for preventive measures.

Yang Junbin [12], in 2022, in response to the monitoring and rectification of extremely steep and hazardous rock masses along the Chongqing-Huaihua Railway, advocated for a wall-following aerial survey strategy with drones. This enhanced the accuracy of three-dimensional real-world models, effectively supporting rock mass monitoring and design of remediation works, thereby achieving high-efficiency, low-cost, and low-risk construction.

4. Ground-Level Surveillance by Drones

4.1. Track Inspection Operations by Drones

A pivotal component of railway infrastructure maintenance, drone-based track inspections furnish critical data for predictive risk assessment and preemptive interventions. Regular inspections are indispensable for ensuring the safety and efficiency of train operations. Traditionally, inspections involved manual visual checks by personnel walking alongside the tracks or utilizing track vehicles, which were time-consuming, subjective, and highly dependent on observer expertise. The emergence of intelligent inspection techniques has since revolutionized this landscape.

In 2019, Yunpeng Wu [13] addressed the issue of degraded image quality due to atmospheric conditions like fog or haze during drone-based railway infrastructure inspections. He proposed a novel end-to-end dehazing network for railway images acquired via drones, focusing on optimizing network architecture and loss function design. Compared to conventional approaches, the algorithm demonstrated superior performance.

Liu Cheng [14] et al., in 2018, integrated drone technology with image processing techniques to identify railway tracks from aerial images. This involved filtering, connected component detection, edge detection, and line detection to isolate the track area. In 2019, Milan Banić [15] underscored the potential of drones for robust and reliable visual track inspections, leveraging edge detection and feature extraction algorithms to pinpoint track locations.

Zhu Pengcheng [16] explored the impact of different ground control point configurations on the accuracy of drone-monitored railway deformation in mining areas in 2021. Utilizing a 3D point cloud model, this study provided new technical means for drone-based monitoring of linear ground features.

4.2. Drone Surveillance of Railway Surroundings and Environment

Beyond the confines of railway track inspections, UAVs have emerged as a versatile tool for monitoring the broader railway environment. The complexity and scale of adjacent facilities, structures, topography, and environmental factors necessitate a surveillance solution that overcomes the limitations of traditional manual inspections, characterized by their arduosity, inefficiency, lengthy intervals, and susceptibility to natural constraints.

In 2021, Huang Yixin et al. addressed the challenge of identifying structures in the vicinity of railway lines using convolutional neural networks (CNNs). Recognizing the deficiencies of the U-Net architecture, particularly the loss of feature information during downsampling and its large parameter footprint, they introduced an improved U-PPM series network for building extraction. Experimental results demonstrated notable improvements in recognizing irregularly shaped buildings; however, the complexity and heterogeneity of real-world data necessitated enhancements to the model's generalizability.

Yundong L in 2020, aiming to optimize the detection of small objects in railway environments, proposed a multi-block SSD methodology. This algorithm segmented the original image into several overlapping sub-images, augmenting subscale detection and enriching local contextual information. Particularly beneficial in scenarios with limited samples, this approach significantly boosted target detection accuracy. Notably, the algorithm achieved a 23.2% increase in the detection rate of small objects.

5. Conclusion

This paper has encapsulated the state-of-the-art applications of unmanned aerial vehicle (UAV) visual technologies in the inspection of railway overhead contact lines, bridges, tracks, embankments, adjacent geological hazards, and surrounding environments within China's railway network. It has become evident that the diversity in UAV models, data processing software, and inspection methodologies presents a significant challenge to standardizing inspection procedures and managing operations efficiently.

The inconsistency in technical specifications and operational protocols has compounded the difficulties in implementing systematic and regulated UAV inspections. Moreover, quality disparities among UAVs contribute to higher accident rates, prompting railway authorities to impose strict regulations prohibiting the operation of drones, airships, and balloons within a 500-meter radius of railway lines. Consequently, the imperative for future research lies in the development of specialized UAV inspection techniques that are congruent with railway-specific requirements, aiming to achieve uniformity and standardization in UAV inspection practices.