1. Introduction

Environmental perception technology has always been widely regarded as one of the core functions of autonomous driving systems, and its performance directly affects the safety and reliability of autonomous driving systems. In recent years, the combination of multi-sensor fusion technology and deep learning algorithms has driven significant advancements in environmental perception technology [1]. However, three critical issues remain unsolved: addressing the impact of extreme weather, defending against cyberattacks, and resolving network security risks. This article will focus on the challenges and future development trends of information acquisition sensor technology, sensor fusion, and environmental perception technology, focusing on autonomous driving environmental perception technology. This paper is divided into multiple levels: comprehensive vision sensors, millimeter-wave radar sensors, and other sensors, emphasizing the related applications and roles of multi-sensor fusion technology, while paying attention to the problems faced by environmental perception technology in some special environments and its inherent limitations, and in-depth analysis. The research results of this paper provide theoretical and technical support for the construction of a low-cost, high-reliability, and all-weather environmental perception system.

2. Overview of autonomous driving environmental perception technology

Autonomous driving environmental perception technology refers to the technology that enables vehicles to independently, in real time, and accurately acquire and interpret surrounding environmental information through sensors such as vision, infrared, and radar, combined with specific algorithms. It is not only the basis for autonomous vehicles to achieve autonomous navigation, obstacle avoidance, path planning and other functions, but also the key to ensuring driving safety and improving driving efficiency [2].

However, the autonomous driving environment is complex and dynamic, and sensors are greatly affected by factors such as weather, lighting, and occlusion, and the perception system faces a large number of technical and safety challenges. To address these challenges, researchers have been exploring new sensor technologies and data processing methods (Table 1).

Autonomous driving perception uses a variety of sensors to collect surrounding environmental information and provide a basis for subsequent decision-making. Commonly used sensors mainly include cameras, lidar, and millimeter-wave radar. Various sensors have their own advantages and disadvantages in terms of detection principle, information type, adaptability to the working environment and cost. Reasonable integration of multiple sensors is the key to enhancing the overall perception performance of the system.

2.1. Camera

Cameras are one of the most commonly used vision sensors, providing high-resolution images, and collecting rich texture and color information. They are indispensable for core environmental perception tasks such as object type identification, traffic sign reading, and lane line detection. Common application scenarios include lane line detection, vehicle classification, and traffic light recognition. Moreover, the relatively low cost and easy installation required for the camera also make it a mainstream sensing solution.

However, the camera has the disadvantages of light sensitivity, lack of depth information, and lack of direct ranging ability and cannot directly obtain the depth information of the collected object, which limits its usability in complex environments.

2.2. Li DAR

LiDAR (Light Detection and Ranging) is a radar system operating in the light wave frequency band, classified as an active detection technology capable of accurately and rapidly acquiring 3D spatial information of ground objects or the atmosphere. LiDAR emits a laser to the target object, and then determines the actual distance of the target object according to the time interval between reception and reflection, and then obtains the position information of the object through geometric relationships according to the distance and the angle of laser emission. In addition, the motion speed, attitude, and shape of the target object can be determined by analyzing the signal strength and frequency changes of the reflected laser. It is commonly used to construct obstacle contours and pavement elevation models.

LiDAR has extremely high resolution, and strong anti-interference ability, and the amount of information it obtains is also very rich, which can directly obtain the target's distance, angle, reflection intensity, speed and other information to generate multi-dimensional images of the target. In addition, LiDAR supports 24/7 operation, and the laser actively detects without relying on external lighting conditions or the radiation characteristics of the target itself. However, lidar still faces limitations such as high cost, reduced point cloud density in rain, and snow/dust weather, and ranging failures caused by low-reflectivity objects such as sunshades.

2.3. Millimeter-wave radar

Millimeter wave radar detects the distance, speed, and angle of objects by sending high-frequency electromagnetic waves and measuring the frequency, time delay, and phase change of the echo. Its main advantages are that the detection distance is long, up to over 200 meters; it has good detection performance; it is insensitive to color and temperature; it has a fast response speed; it has strong adaptability; it can still work normally in rain, snow, fog, and other severe weather; and its anti-interference ability is strong. It is often used for moving target tracking in high-speed scenarios. Traditional millimeter-wave radar has low angular resolution; it is difficult to distinguish between dense pedestrians or fine contours, and it is impossible to accurately model surrounding obstacles, which is a big problem when driving autonomously, and its coverage area is fan-shaped, it has blind spots, and it cannot recognize traffic signs and signals. Radar point cloud data is becoming an important supplement to benchmark LiDAR in 3D perception, especially in safety-critical autonomous driving scenarios. Radar simulation and calibration technology is also advancing to ensure the efficient and accurate perception performance of radar [3].

2.4. Ultrasonic radar

Ultrasonic radar works by transmitting ultrasonic waves via an ultrasonic emitter, then calculating the distance to the target based on the time difference between transmission and reception of the reflected waves. Ultrasonic radar can be waterproof and dustproof; even if there is a small amount of sediment covering it, it will not be affected. Its detection range is usually between 0.2 meters and 5 meters, but the accuracy is relatively high, so it is very suitable for parking.

3. Multi-sensor fusion technology and applications

A single sensor is limited by field of view, weather, or resolution, and multi-sensor fusion has become an inevitable choice for achieving all-weather and all-target accurate perception.

Multi-sensor fusion, combined with environmental perception, gives autonomous vehicles the ability to observe and understand their surroundings. Specifically, environmental perception technology is like the vision system of an autonomous vehicle, using a variety of sensors such as cameras, millimeter-wave radar, and lidar to capture and analyze detailed data in the environment, such as lane markings, traffic light status, pedestrian dynamics, other vehicles’ movements, and potential obstacles. The real-time data provided by these sensors provides critical inputs for decision-making systems in autonomous vehicles [4].

3.1. Multi-vision sensor fusion

In automotive intelligent driving, through multiple monocular cameras with different focal lengths and different elevation angles, the detection and recognition capabilities of traffic signs and various road signs in different positions can be obtained. For example, a telephoto camera can capture, the traffic lights at 100 meters, which are large enough, and the colors on the traffic signs at 100 meters are clearly visible. In contrast, a short-focus camera (wide-angle camera) cannot clearly capture the numbers on traffic signs 100 meters away, but it can acquire wider-range close-range environmental information. Therefore, the combination of multiple monocular vision sensors is widely used in the field of intelligent connected vehicles. Moreover, the rich image information and depth data provided by multi-vision sensors can help build more accurate 3D environmental models and support autonomous driving path planning and decision-making. For example, the ApolloScape dataset uses multi-camera linkage to achieve high-resolution dense semantic point cloud annotation, thereby improving the accuracy of semantic segmentation and 3D reconstruction [5].

3.2. Camera and radar fusion

Camera and radar fusion is widely used in vehicle detection and ranging tasks. The camera makes it easy to detect the vehicle in the image, but the ranging accuracy of the monocular camera is relatively low, and the ranging of the binocular camera is more complicated. Radar, on the other hand, is the opposite, which makes it easy to obtain the distance information of the obstacles in front of them but struggles to identify obstacle types. Neither of them can complete the detection and ranging of the vehicle alone, but after multi-sensor fusion through spatial alignment and temporal alignment, the camera is used to detect the vehicle, and the radar is used to obtain distance information. For example, the Center Fusion model uses a network based on the center point to fuse radar and camera data to effectively improve the accuracy and speed of object 3D detection [6].

4. Challenges of autonomous driving environmental perception technology

4.1. The impact of complex environment and changeable weather conditions

The sensor performance is significantly affected by weather environments such as light, rain, snow, fog, and shadow, resulting in shortened detection distance, increased noise, and higher false recognition rates. For example, cameras are severely interfered with by low light and strong backlight, LiDAR point clouds perform unstably in rainy and snowy weather, and millimeter-wave radar is relatively more robust but has lower resolution [7,8].

Effective mitigation strategies include three key directions: developing weather-resistant sensors, optimizing sensor data preprocessing (e.g., noise filtering, point cloud artifact removal), and leveraging multi-sensor fusion to enhance the system’s anti-interference capability [8].

4.2. Aware of the network security threat of the system

Autonomous driving systems are cyber-physical systems that are vulnerable to physical and cyber attacks. For example, camera spoofing, lidar deception, millimeter-wave radar jamming, GPS spoofing and other means can lead to erroneous judgments of the environmental perception module during the autonomous driving process, which will seriously threaten the safety inside and outside the car.

The network security threat of environment-aware systems has evolved from the deception of a single sensor to a coordinated attack of the entire 'sensor-communication-algorithm-data' chain. In the future, it will be necessary to combine functional safety and information security standards to build a defense-in-depth system from chip to cloud, build a complete security threat modeling and risk assessment framework, improve the security protection capabilities of perception systems and machine learning models, and design a multi-layered defense system.

5. Conclusion

With the in-depth advancement of research of autonomous driving environmental perception technology, the development potential of this field has become more and more prominent. While autonomous driving environmental sensing technology has made significant progress, it still faces many challenges. The impacts of complex environments and variable weather conditions, as well as the cybersecurity threats to perception systems discussed in this paper, are only a subset of these challenges, but they are also among the most critical. The development of autonomous driving environmental perception technology must focus on improving perception accuracy, enhancing environmental adaptability and multi-sensor fusion.