1. Introduction

The rapid advancement of wireless communication technologies is precipitating substantial shifts in the manner by which individuals establish and sustain their daily connections, as exemplified by the emergence of 5G and the forthcoming 6G networks. Radio Resource Management (RRM), a major challenge for wireless communication systems, aims to optimize the utilization of limited wireless resources (such as spectrum and power) to enhance network performance and satisfy user demands. Nevertheless, traditional approaches to wireless resource management are somewhat constrained in large-scale, complex, and dynamically changing network environments. These approaches have long formally developed design goals and constraints as optimization problems, which are subsequently solved to some extent using mathematical techniques. Despite the effectiveness of these methods in most network environments, they may not be optimal in complex, dynamically changing network environments. In contrast, machine learning has seen significant advancements in terms of adaptability, robustness, scalability, and predictive ability, particularly in the context of 6G [1][2]. The integration of wireless communication networks and machine learning is regarded as a promising avenue for future development. The existing research has focused on specific aspects of deep learning in resource management, but lacks a comprehensive overview of developments over recent years.

This paper examines three algorithms that are currently being utilized in the domain of wireless resource management, outlines the fundamental principles of each algorithm, and explores the advantages, research progress, and future challenges of each algorithm, which aims to demonstrate a comprehensive understanding of how these algorithms can optimize wireless resource management. Through a comprehensive account of existing research and practical applications, this paper provides a valuable reference for further research on the application of machine learning in resource management for wireless communications. Therefore, the paper provide insights into the capabilities and limitations of various algorithms so as to promote research progress and practical applications in this area. In short, it will facilitate the development of more efficient and adaptive network solutions.

2. Overview of Machine Learning Methods

2.1. Graph Neural Networks

Graph neural networks, a class of neural networks specialized in processing graph-structured data, such as social networks, and knowledge graphs, achieve information transfer through the product of the pro-order matrix A and the graph signal X. And the core idea is to take advantage of the relationship between nodes and edges in the graph structure, and to better capture the global and local features of the graph through successive iterative updating of each node, where each node receives the information generated by the neighboring nodes information from neighboring nodes and update itself with it. Mathematically, the convolution operation of GNN can be expressed as (where D means the degree matrix of A, and W represents the learnable weight matrix):

$Z=F(A,X)=D^{−1}A\ X\ W$ (1)

GNNs are currently widely utilized in a variety of domains, including knowledge retrieval, molecular chemistry, traffic networks, and wireless resource allocation. Nevertheless, despite the obvious advantages of graph neural networks in the processing of graphs, and the good scalability of some irregular data structures, there are still some problems. Specifically, due to the high computational complexity, requires a long computation time and more arithmetic support; secondly, after many runs, graph neural networks also suffer from the problem of information loss.

2.2. Deep Reinforcement Learning

Deep reinforcement learning, a machine learning algorithm, combines two techniques, deep learning and reinforcement learning, where deep neural networks can handle many complex data structures, while reinforcement learning can interact with the environment and learn strategies to maximize long-term returns. Integrating the features of both, deep reinforcement learning is able to handle high-dimensional states and spatial actions, which allows it to excel in complex environments. The fundamental tenet of deep reinforcement learning algorithms is to enhance an agent's decision-making process through approximation and value-based strategies, encompassing value-based methods and policy gradient methods. Deep Reinforcement Learning is widely used in game AI, robot control, automatic driving, etc., and has been used in a mature way. In wireless resource management, deep reinforcement learning has also been applied in many aspects, such as spectrum allocation, power control and load balancing, and has better performance than traditional algorithms in these aspects.

2.3. Federated Learning

Federated learning, a novel cryp-tographic distributed machine learning algorithm that has emerged in recent years, which allows multiple participants to jointly train a global model of artificial intelligence without the data leaving the local area. The algorithm works by initializing a global model at a central server, then dividing the training tasks of the model locally and training them, uploading the updated parameters to the central server after the training is completed, and finally employing an aggregation algorithm to integrate these parameters into a new global model. By iterating the above steps, the final model converges [3]. In recent years, federated learning has been widely used in various domains such as smart cities, smart healthcare, smart transportation, smart grids and social media because of its data privacy protection, less bandwidth requirement and better model performance [4][5].

Federated learning also has some drawbacks in practical applications, for example, the data of the participants may not be independently and identically distributed (Non-IID), which affects the stability of the model training and the convergence speed, the system needs more complex mechanisms to manage multiple participants, and the frequent transmission of the model parameters brings more communication overhead, which need to be improved by future research.

3. Applications of Machine Learning in Resource Allocation

3.1. Graph Neural Networks in Resource Allocation

The power control is of paramount importance in all wireless communication systems, given the intricate relationships and interactions between a multitude of nodes. Traditional approaches are typically founded upon convex optimization methods. However, a considerable number of issues are non-convex, and the application of fixed-parameter algorithms to identify the optimal control method and scale crosswise is often challenging. In contrast, GNNs are capable of capturing the potential non-linearities and complex dynamics within the network with greater efficiency, as they represent the nodes and their connectivity relationships as graph structures

In power control, GNNs outperform other algorithms on specific problems. Based on the algorithmic alignment framework [6], it is demonstrated that GNNs outperform multilayer perceptrons (MLPs) for the K-user interference management problem, and that the performance gap between these two methods increases with K [7]. In the field of radio resource management, GNNs also demonstrate efficacy. Yifei Shen et al. verified the equivalence between MPGNN and a series of distributed optimization algorithms, and accordingly analyzed the performance and generalization ability of MPGNN-based methods. Extensive simulation results using power control and beamforming as examples show that the proposed method, trained in an unsupervised manner using unlabelled samples, matches and even outperforms classical optimization algorithms without domain-specific knowledge in some cases. In addition, the proposed method is highly scalable, which takes only 6 ms to solve the beamforming problem in an interfering channel with 1000 transceiver pairs on a single GPU. This shows that MPGNNs not only possess the advantages of high efficiency and scalability, but also provide theoretical reliability and interpretability in solving large-scale wireless resource management problems [8].

3.2. Deep Reinforcement Learning in Resource Allocation

In recent times, the field of physical layer design and resource allocation has witnessed a surge in interest with regard to interference management in multicellular networks. The deployment of a large number of base stations (BSs) covering an area of point-to-point wireless connections results in the formation of multi-cell interference systems [9]. Traditionally, numerical optimization methods have been the primary means of solving resource allocation problems [10], including Weighted Minimum Mean Square Error, Fractional Planning, and Interference Pricing. Typically, these methods employ key performance metrics, such as channel realizations, as inputs through iterative algorithms and output an optimal power allocation strategy. However, most of these optimization problems are challenging to resolve due to their non-convexity and high-dimensional parameters.

At present, deep reinforcement learning techniques have been employed to address wireless resource management challenges, demonstrating significant advantages in scenarios where traditional methods are inadequate. n addition, the method has been applied to a variety of areas such as spectrum access, throughput maximization, power and channel allocation, thereby enhancing the performance of 5G wireless networks. The two basic DRL algorithms, Q-learning and DQL, represent policy-based and value-based design techniques, respectively. The classical Q-learning technique has been used for downlink resource allocation in Non-Orthogonal Multiple Access (NOMA) networks [11], while DQL has been used to solve problems that cannot be handled in traditional learning systems. For example, in Cloud RAN, Xu Z et al. used DQL for power allocation to reduce power consumption and ensure per-user reliability constraints [12]. Besides, Li X et al. applied DQL to a distributed spectrum sharing scheme to facilitate resource allocation to multiple users in a non-cooperative manner [12].The experimental results show that with, the help of learned power control policies, an auxiliary user is capable of intelligently adjusting its transmit power so as to reach the target state from any initial state within a few transition steps.

3.3. Federated Learning in Resource Allocation

In wireless resource management, an efficient resource management and device scheduling scheme is of paramount importance, which enables other devices to participate in the learning process, thus improving the overall learning performance. Existing federated learning algorithms primarily concentrate on resource optimisation [13], through which the communication burden of joint learning in wireless networks can be reduced, based on which, Zhixiong Chen et al. proposed Partial Model Aggregation-FL (PMA-FL), designed to reduce the amount of data in the transmission phase and to improve the learning performance in heterogeneous local data distribution scenarios. Experimental results show that the proposed FL algorithm achieves faster convergence and higher model accuracy than the benchmark scheme, with accuracy improvements of 3.13% and 11.8% on the MNIST and CIFAR-10 datasets, respectively. Furthermore, on the MNIST dataset, the proposed joint equipment scheduling and resource management algorithm reduces the energy budget by approximately 29% and the time budget by 20%, while achieving higher accuracy than the benchmark. On the CIFAR-10 dataset, the proposed algorithm achieves slightly higher accuracy than the benchmark solution and reduces the energy budget by 25% or the time budget by 12.5% [14].

4. Future Outlook and Recommendations

In the future, machine learning algorithms will play an increasingly significant role in the allocation of wireless communication resources. As 6G communications evolve, machine learning will provide comprehensive assistance in a range of areas, including network orchestration and management, physical layer signal processing, data mining, and service-oriented context-aware communications. In particular, the development of deep learning techniques will significantly enhance the performance and reasoning speed of communication systems through the application of deeply unfolded signal processing methods. In this context, Multi-Intelligent Deep Reinforcement Learning (MADRL) will become an effective tool to cope with dynamic resource allocation in complex wireless network environments. MADRL enables the system to perform efficient resource allocation and management in response to changing network conditions and user demands. Besides, edge intelligence will assume a pivotal role in future 6G networks, facilitating the deployment of large-scale IoT device-dense networks and markedly enhancing the flexibility and efficiency of resource allocation [15].

To facilitate advancement in this field, interdisciplinary research will become a crucial undertaking. The combination of knowledge from multiple disciplines, including machine learning, signal processing, and communication engineering, can facilitate the development of more advanced and efficient resource allocation algorithms. Furthermore, the scalability and efficiency of the algorithms must be a primary focus to meet the demand for large-scale and high-density deployment in future communication networks. In addition, ensuring the security and interoperability of algorithms in resource allocation represents an important direction for future research.

5. Conclusion

This paper presents a summary of the application of deep reinforcement learning, federated learning and graph neural networks in wireless resource management. A review of the current research findings indicate that these techniques show great potential in improving the performance of wireless networks and solving complex resource management problems. Specifically, the deep reinforcement learning technique excels in spectrum access, throughput maximization, power and channel allocation, which is able to effectively deal with non-convex optimization problems that are difficult to be solved by traditional methods. The federated learning technique achieves the objective of co-training a global model with multiple participants while preserving data privacy, thus reducing bandwidth requirements and enhancing model performance. The graph neural networks demonstrate their benefits in resource allocation and optimization problems when confronted with complex network structure data. However, these algorithms still exhibit certain limitations. Future algorithms should enhance their environmental adaptability and multimodal multi-task learning ability, and combine with traditional algorithms to develop more sophisticated models. Through continuous research and innovation in these directions, the application of machine learning algorithms in wireless resource management will become more efficient, intelligent, and reliable, thus promoting the development of future wireless communication networks.