1. Introductory

The rapid advancement of communication technology has led to significant challenges for conventional receivers in signal detection. These challenges include an increase in signal types, the complexity of the channel environment, and the requirement of higher transmission rates and bit error rates. In the context of a multitude of input signals with diverse modulation and coding techniques, the detection accuracy and performance of conventional receivers are negatively impacted. Furthermore, traditional receivers are constrained by their limited ability to handle a wide range of carrier frequencies and modulation modes, which impairs their flexibility. Their anti-jamming capabilities are inadequate in the presence of strong interference or complex electromagnetic environments, and their utilization of spectrum resources is suboptimal, leading to constrained system capacity and transmission rates. In recent years, the development of artificial intelligence (AI) technology has opened up new avenues for the integration of communication receivers and AI. Deep learning, a subfield of AI, employs the construction and training of deep neural networks to effectively predict or classify results by learning complex patterns and features from a large amount of data. In the field of communication, the incorporation of deep learning in receivers can significantly enhance the accuracy and flexibility of signal processing. Figure 1 illustrates the intelligent receiver model resulting from the integration of a deep learning neural network model with a conventional receiver.

Nevertheless, there are several limitations associated with deep learning-based intelligent receivers. These include high algorithmic complexity, a limited scope of application [1], a lengthy training time for network models, a high threshold for practical application, and so forth. There is, therefore, considerable scope for further improvement. At present, the combination of intelligent receivers with multiple-input multiple-output (MIMO) systems, chaotic communications, and non-orthogonal multiple access (NOMA) technology has become a subject of considerable research interest. As the fifth generation of mobile communication continues to develop, the MIMO system is becoming increasingly sophisticated. It is anticipated that AI will be incorporated into the MIMO system to enhance the performance of the communication system [1-3]. In the field of chaotic communication, intelligent receivers can learn and extract the correlation of received signals, thereby improving the reliability of demodulation [4]. The integration of intelligent receivers into NOMA systems has been demonstrated to enhance both the BER performance and system robustness. This paper will present a summary and analysis of the research related to intelligent receivers based on deep learning, as well as a prediction of its future development prospects. The remainder of this paper is organized as follows: Chapter 2 introduces the theoretical foundation of wireless communication and neural networks. Chapter 3 describes the research progress of intelligent receivers based on deep learning. Chapter 4 introduces the practical application of intelligent receivers. Chapter 5 discusses the opportunities and challenges of intelligent receivers and looks forward to their future development.

Figure 1. Schematic diagram of the intelligent receiver	Figure 2. Block diagram of digital communication system structure

2. Theoretical foundations of wireless communications and neural networks

2.1. Wireless communication systems

Communication represents a crucial instrument for the exchange of information within human society. In light of the ongoing evolution of technology, a plethora of communication modalities have emerged. These include wired communications, such as fiber-optic communications, coaxial cables, and telephone lines, and wireless communications, such as satellite communications and cellular mobile communications. Wireless communication systems can be classified into two categories: analog communication systems and digital communication systems. This classification is based on the type of information transmitted. As illustrated in figure 2, a typical digital communication system, for instance, comprises three principal components: transmitter, channel, and receiver, as delineated below.

2.1.1. Transmitters

The transmitter is primarily responsible for the processes of coding, encryption, and modulation of the source signal, which are employed to convert it into a signal that is suitable for transmission over the channel. In a digital communication system, the transmitter comprises the following components: the source, source coding, encryption, channel coding, and digital modulation. The particular configuration is illustrated in Figure 2.

2.1.2. Signal path

The term "channel" is used to describe the path or route that a signal takes as it is transmitted from the transmitter to the receiver. There are two main types of channels: wired and wireless. A transmission channel is the medium through which a signal is conveyed from a transmitter to a receiver. Transmission channels can be classified as either wired or wireless. In mobile communications, the presence of noise sources in the channel may result in the incorrect reception of the signal.

2.1.3. Receivers

The principal function of the receiver is to receive the signals transmitted over the channel and perform processes such as demodulation and decoding, thereby restoring the signals to their original information. The receiver structure depicted in Figure 2 comprises five principal components: digital demodulation, channel decoding, decoding, source decoding, and signal lodging.

2.2. Neural networks and deep learning

A neural network is a computational system inspired by the biological nervous system and consists of a large number of interconnected artificial neurons. In contrast, deep learning is a machine learning method that employs deep neural networks for the purposes of learning and model construction. As the foundation of deep learning, neural networks achieve complex tasks through multi-level information transfer and learning. As deep learning is based on neural network models, the following section will describe several typical neural network models.

2.2.1. Convolutional neural network (CNN)

CNN is a feed-forward network, which is particularly suited to the processing and analysis of image data. In a feed-forward network, each neuron is divided into different groups according to the order of receiving information. Each group is regarded as a neural layer, and the information is passed down through the neural layers layer by layer. The information of the whole neural network is passed in one direction, without reverse transmission. A convolutional neural network is structured with an input layer, a convolutional layer, a pooling layer, and a fully-connected layer, which are stacked on top of each other. A number of models are currently in use, including AlexNet, GoogleNet, ResNet, and others. The specific structure of convolutional neural networks is described in the following section.

2.2.2. Recurrent neural network (RNN)

RNN is a class of neural networks designed for processing sequence data. In contrast to feed-forward neural networks, RNNs possess a memory function for sequences through recurrent connections, whereby the input of the previous step is treated as the input of the current step. RNNs are predominantly employed in the domains of natural language processing, time series processing, and speech recognition. Figure 3 depicts the fundamental structure of an RNN, which comprises an input layer, a hidden layer, and an output layer. In RNN, the data from the previous hidden layer can be trained in the subsequent hidden layer, in conjunction with the data from the subsequent input layer.

Figure 3. RNN network structure

The original RNN is susceptible to gradient explosion and gradient vanishing when processing long sequence signals. Consequently, researchers have proposed variants of RNN, with the most notable examples being Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU).

2.2.2.1. LSTM

LSTM is a specific type of recurrent neural network, comprising Input Gate, Forget Gate, and Output Gate, as illustrated in Fig. 4. The distinctive gating structure of LSTM enables it to overcome the limitations of traditional RNNs in processing long sequences, addressing the challenges of gradient vanishing and gradient explosion. This has led to LSTM's success in a range of tasks requiring the memorization of long-term dependencies.

Figure 4. LSTM structure Figure 5. GRU structure

2.2.2.2. GRU

GRU is a neural network model for processing sequence data. It consists of a reset gate, an update gate, and a simpler gating mechanism than LSTM. GRU performs better in tasks that are computationally efficient and insensitive to time dependence. It is suitable for some short-term time series prediction tasks. Figure 5 illustrates the fundamental structure of GRU.

2.2.3. Graph Neural Network (GNN)

A GNN is a neural network that is designed to learn graph-structured data and uncover information such as features in graph-structured data. In the field of signal detection and demodulation, the article [5] proposed the Adaptive Visibility Graph (AVG) algorithm, which maps a time series into a graph and employs a graph neural network as a classifier. This is followed by the construction of an end-to-end AVGNET, which assists in signal detection and demodulation.

2.2.4. Emerging Neural Network Models

The aforementioned CNN/RNN/GNN models are limited in their ability to capture local relationships. In contrast, the Transformer model, proposed in 2017, effectively addresses this issue. The core of the Transformer lies in the self-attention mechanism, which enables it to learn long-distance features between data effectively. The fundamental structure of the Transformer model is illustrated in figure 6. The Transformer has been instrumental in propelling the rapid advancement of natural language processing and computer vision. Nevertheless, the current generation of Transformer-based models still necessitates the use of high-end graphics cards with larger memory for training and testing, which significantly constrains their potential for widespread deployment. In order to overcome the aforementioned shortcomings and enhance the efficacy of the models, a plethora of novel models predicated on sparse attention mechanisms or novel neural network paradigms have been proposed. Among these, state-space models (e.g., Mamba) have emerged as a subject of particular interest.

The State Space Model (SSM) is a mathematical model that describes the behavior of a dynamic system. It employs a set of first-order differential equations (for continuous-time systems) or difference equations (for discrete-time systems) to represent the evolution of the internal states of the system and another set of equations to describe the relationship between the states and the outputs of the system. The SSM allows for parallelism in training and applies a recursive form of linear complexity reasoning.

Figure 6. Transformer structure Figure 7. Mamba structure diagram

In contrast to Transformer, Mamba employs a linear scalability of the sequence length of the state-space model, while simultaneously implementing the modeling capabilities of Transformer. The latter's structure is depicted in figure 7. By means of selective SSM, Mamba is capable of filtering irrelevant information in order to facilitate the processing of sequences in an efficient manner. From a hardware perspective, Mamba employs optimized parallel algorithms for hardware. This enables Mamba to effectively increase the computation rate and reduce the memory requirement. Concurrently, Mamba is devoid of conventional attention and multilayer perceptron (MLP) modules, a simplification that enables the Mamba architecture to scale with the sequence length.

3. Advances in deep learning-based intelligent receivers

The preceding chapter established a theoretical foundation and existing research on the traditional receiver. This chapter will examine the current state of the traditional receiver, identifying areas for improvement.

The traditional receiver has several limitations. It studies a single type of signal, typically serial demodulation and decoding. Its configuration is fixed, limiting its flexibility. It is unable to adapt to complex interference. Its accuracy in detecting signals is also a significant area for improvement. Consequently, research pertaining to intelligent receivers based on deep learning has garnered increasing interest from researchers. This chapter will provide a comprehensive overview of the research progress related to deep learning-based intelligent receivers from multiple perspectives. The aim is to offer a valuable reference for future research, as described below.

Figure 8. LDAMP layer l architecture	Figure 9. Depicts the structure of the GRU-ICI-CSK signal detection algorithm

3.1. Channel estimation

Channel estimation is the process of measuring and estimating the characteristics of a channel at the receiver side with the objective of obtaining channel parameters for signal processing and compensation. Conventional channel estimation methodologies posit that the receiver possesses absolute Channel State Information (CSI), yet in reality, this assumption is challenging to achieve due to noise and guide frequency contamination.

The majority of existing methods for enhancing detection performance typically result in a significant increase in computational complexity. A new generation of deep learning-based channel estimation methods has emerged, which can enhance detection performance while reducing system complexity. In MIMO systems, the uplink and downlink channels of Frequency Division Duplexing (FDD) systems exhibit weak reciprocity due to disparate operating frequencies, rendering it more challenging for the base station to obtain the channel state information (CSI). In contrast, the article [6] employs an intelligent receiver to map the uplink cluster information fed back to the base station by the mobile user, subsequently obtaining the downlink channel CSI. He et al. proposed a denoising-based approximate message passing (LDAMP) network based on the approximate message passing (AMP) algorithm. Each layer of the LDAMP network comprises a feed-forward denoising convolutional neural network (DnCNN), a scatter estimator, and a correlation coefficient. Figure 8 depicts the architecture of the LDAMP layer [7].

Previous studies have proposed the LampResNet network, which combines an iterative AMP algorithm with a residual learning network (ResNet) to address the channel estimation challenge for millimeter-wave massive MIMO in the beam domain [8]. In the field of mobile communications, where high-speed moving objects are subject to Doppler shift during signal reception, the SBGRU estimator introduced in the literature [9] integrates the RNN architecture with the sliding window mechanism, thus demonstrating excellent robustness in dealing with variable-length transmitted symbols, diverse frequency-guided overheads, and changing channel statistical characteristics.

3.2. Signal detection

The objective of signal detection is to distinguish between the original signal and the received signal. As the modulation and transmission of signals become increasingly complex, the interference and noise encountered in the transmission process exhibit randomness, thereby increasing the uncertainty of signal transmission. The limitations of traditional receivers in terms of accuracy and efficiency of signal detection make it challenging to meet the requirements of signal detection in modern communications. Consequently, contemporary research is actively investigating novel methodologies for integrating signal detection technology with deep learning. In the field of wireless communications, deep learning-based receivers are capable of extracting correlation features between received signals, thereby enhancing signal detection from noise. An OAM-DeepSIC detector has been proposed for the purpose of detecting orthogonal angular momentum signals in the presence of mixed modal interference. This intelligent receiver employs a hybrid approach, integrating the strengths of model-driven and data-driven techniques, to achieve a notable reduction in the false symbol rate [10]. In wireless communication, the use of chaotic signals for modulation can enhance the anti-interference performance of communication systems. However, the energy detection scheme of chaotic signals at the receiver side has limitations on Rayleigh channels. To address this problem, Zou et al. proposed a GRU-aided initial condition index chaotic keying system (GRU-aided initial condition index CSK system, GRU-ICI-CSK). The fundamental configuration of the GRU-ICI-CSK system is illustrated in figure 9.

In comparison to the conventional detection methodology, this approach exhibits robust learning and classification abilities, rendering it an efficacious solution for low-power, short-range wireless transmission [11]. Direct Sequence Spread Spectrum (DSSS) systems are employed extensively in civil and military communications due to their exceptional anti-jamming capabilities and low interception rates. Nevertheless, DSSS is subject to limitations in terms of power spectral density and detection. The article[12] proposed two distinct detection schemes: a convolutional neural network (CNN)-based DSSS detection scheme and a hybrid CNN-correlation (CORR) based DSSS detection scheme. The results demonstrate that both signal detection schemes outperform the autocorrelation-based detection approach. The detection of weak signals has consistently been a significant challenge in the field of signal detection. Traditional blind detection methods for weak signals have demonstrated limited effectiveness in low signal-to-noise ratio conditions. In response, Dong et al. proposed a spatial spectrum-based LSTM weak signal detection approach. The efficacy of this approach is demonstrably superior to that of radial basis function neural network, max-min eigenvalue, and energy detection[13].

3.3. Signal Modulation Classification

Deep learning is a widely utilized technique in the field of automatic modulation classification (AMC), which has been demonstrated to enhance the accuracy of AMC.

Figure 10. Overall process of AMC based on deep learning

In contrast to traditional likelihood and feature-based methods, deep learning-based AMC is capable of classifying complex data with greater accuracy and does not necessitate a significant investment in human resources [14-15]. The article [15] proposed an automatic modulation classification relation network (AMCRN) for the study of AMC with small samples. Kim et al. proposed a convolutional neural network (CNN) based hybrid deep learning of signals and images and applied it to signal modulation classification. The model exhibits an enhancement in signal-to-noise ratio and prediction accuracy in comparison to the edge convolutional neural network, autocorrelation bootstrap network, and lightweight convolutional neural network[16].

3.4. signal Demodulation

In the context of communication systems, signal demodulation is the process of extracting the original information signal from the received modulated signal. Multiple Quadrature Amplitude Modulation (M-QAM) is a modulation technique that is widely used in digital communication systems due to its high spectral efficiency, which is limited only by the channel noise level and linearity. While conventional M-QAM demodulation employs coherent demodulation, the article [17] proposes to utilize the robustness of neural networks to demodulate M-QAM signals. Signal demodulation in the field of chaotic communication is also a topic of significant interest in contemporary research. A proposed method for demodulation in a multilevel code-shift M-variant differential chaotic keying system involves the construction of multiple recursive LSTMs and their connection in series with multiple FCLs to form a Deep Neural Network (DNN). This approach is intended to facilitate the recovery of information[18].The fundamental configuration of the DNN-aided intelligent receiver is illustrated in figure 11

Figure 11. Smart DNN-assisted receiver structure	Figure 12. DNN structure oriented to smart detectors

In contrast to the conventional single-mode AMC based on CNNs, Qi et al. proposed a Waveform-Spectrum Multimodal Fusion (WSMF) method based on Deep Residual Network (DRN) with the objective of achieving the fusion of multimodal information from multiple transform domains in order to obtain a more discriminative feature. In comparison to the conventional CNN-based AMC approach that utilises unimodal information, WSMF demonstrates enhanced performance in the context of higher-order digital modulation types[19].

3.5. Signal Decoding And Recovery

The process of decoding is the restoration of a signal or data from a coded form to its original information. In MIMO systems, article [20] proposed a DNN-based receiver architecture for spatial media modulation—MIMO (SMBM-MIMO) systems, which is detected in an end-to-end manner to recover the transmitted symbols directly with an offline training process.

The results demonstrate that this deep neural network (DNN)-based receiver structure exhibits enhanced performance and throughput compared to the maximum likelihood (ML) receiver based on linear minimum mean square error (LMMSE) channel estimation in the context of multiple receiving antennas. In the context of spherical decoding algorithms, related studies have proposed a deep learning-based spherical decoding algorithm, which has the advantage of low computational complexity compared to the traditional spherical decoding algorithm[21]. The joint deep learning approach to signal detection and channel decoding exhibits enhanced learning capabilities, rendering it a promising solution for signal decoding. The article [22] aims to address the joint noncoherent differential detection and channel decoding problem by constructing a deep neural network (DNN) composed of long short-term memory (LSTM) units. This approach is expected to provide faster processing speeds and recovered accuracy.

3.6. Current Status of Research on Intelligent Reception of OFDM Communication Signals

Orthogonal Frequency Division Multiplexing (OFDM) is a multicarrier system for high-speed wireless channels. However, its channel orthogonality is easily lost due to rapidly changing channel characteristics, which results in inter-channel inter-ference (ICI). In order to overcome ICI and improve the reliability of intelligent receivers, two solutions have been proposed: orthogonal frequency division multiplexing with index modulation (OFDM-IM)[23] and direct modulation combined with OFDM-IM (DM-OFDM-IM)[24].The article [25] proposed an index-bit-detection neural network with a modulation parameter-independent structure (IBDNN-MPI), a model that can be configured with different configuration parameter MPs.

3.7. Current status of research on intelligent reception of NOMA communication signals

NOMA has garnered significant interest in the context of fifth-generation and future mobile communication systems. Xie et al. put forth a novel deep learning-assisted receiver for NOMA joint signal detection, which is capable of simultaneously performing channel estimation, equalization, and demodulation functions. In comparison to the conventional NOMA signal detection approach, the utilization of the proposed innovative deep learning receiver for signal processing in the Tap Delay Line (TDL) channel model exhibits enhanced feasibility and resilience [26]. In the article [27], a deep learning-based joint channel estimation and signal detection method is proposed for the multiuser Orthogonal Frequency Division Multiplexing with Non-Orthogonal Multiple Access (OFDM-NOMA) scheme under Rayleigh fading channel conditions. The DL-based detector (DLD) proposed in the article is capable of performing symbol detection based on the guide frequency and data signals without the necessity for additional channel estimation and interference cancellation. This renders it more advantageous in terms of BER performance and robustness. Furthermore, it can be applied to any fading conditions without the need for re-training, which is a promising feature.

4. Intelligent Receivers In Practice

Conventional receivers are constrained by fixed algorithms and inadequate anti-jamming capabilities, which impede their suitability for future communications. Intelligent receivers integrate advanced hardware and intelligent software algorithms to optimize signal processing in complex environments, offering a wide range of potential applications. In light of the advancements in technology, intelligent receivers have become an indispensable component of modern communication systems. The following sections will discuss several major applications of intelligent receivers in practice.

4.1. Wireless Communication Field

In the field of wireless communications, the introduction of smart receivers has the potential to adaptively handle complex channel conditions, thereby enhancing the reliability of data transmission. In the context of the Internet of Things (IoT), the deployment of smart receivers can facilitate the efficient processing of signals from a multitude of communication protocols and dynamic environments, thereby enhancing network connectivity and stability.

4.2. Satellite communications area

In the field of Earth observation, communication between satellites and ground-based stations is a necessity. However, this channel is more complex and susceptible to various external factors, including weather, atmospheric conditions, and other variables. The use of intelligent receivers can enhance the accuracy and reliability of data transmission. In the context of the global satellite navigation system, the deployment of intelligent receivers can enhance signal processing capabilities and improve positioning accuracy.

4.3. Military and defense field

An intelligent receiver can enhance the detection and tracking accuracy of the radar on the target, augment the anti-jamming capability, and, in the context of complex electromagnetic environments associated with electronic warfare, facilitate real-time detection of the other party's communication interference, thereby enhancing the anti-jamming ability.

4.4. Automated Driving and Smart Furniture Sectors

In terms of in-vehicle communication, intelligent receivers can facilitate low-latency, high-reliability in-vehicle communication. Furthermore, intelligent receivers can integrate data from different sensors to enhance the safety and decision-making ability of autonomous driving[28]. Figure 13 presents the fundamental model of the OTFS system in the field of telematics.

In the field of smart home technology, intelligent receivers can enhance the coverage and stability of wireless networks, thereby facilitating seamless operation and connection of various smart home devices. Conversely, the efficient transmission of data can enhance the real-time monitoring capability of home security systems.

Figure 13. Use of OTFS in Telematics

In conclusion, intelligent receivers are capable of adaptively optimizing signal processing and data transmission in complex and changing environments through the use of deep learning and other advanced artificial intelligence technologies, and have broad application prospects. The deployment of these technologies will markedly enhance the efficiency, reliability, and intelligence in a multitude of fields.

5. Summary and outlook

This paper presents a review of the application and research progress of deep learning-based intelligent receivers in wireless communication systems. As communication technology has developed, the amount of information to be processed by receivers has increased exponentially, and the requirements for accuracy and processing speed have become more and more stringent. In parallel, the exponential growth of deep learning technology has made it possible to integrate communication and deep learning. This paper introduces the fundamental concepts of wireless communication systems and deep learning neural networks, which provide the theoretical basis for intelligent receivers based on deep learning technology. Subsequently, Chapter 3 of this paper provides an overview of the current research status of intelligent receivers in the field of signal reception. This includes an examination of the processes of channel estimation, signal detection, modulation identification, demodulation, and decoding. From these studies, it can be concluded that the intelligent receiver is capable of efficient signal reception in the presence of complex interference conditions through the integration of deep learning technology. This technology plays a pivotal role in the advancement of future communication systems.

Nevertheless, the current development of intelligent receivers is still in its infancy. Its algorithmic complexity and application scenarios have certain limitations, the law of learning strategy is not yet clear [29], and part of the smart receiver model is still in the stage of simulation and experimentation. Further research is therefore required to ascertain the applicability of intelligent receivers in actual systems. Consequently, there is considerable scope for further development of intelligent receivers, including their deployment in smart homes, 6G networks and national defense and security. Future research may further optimize the model structure of intelligent receivers and enhance their real-time and robustness in order to cope with more challenging communication scenarios.