1. Introduction
The rapid advancement of wireless communication technologies is precipitating substantial shifts in the manner by which individuals establish and sustain their daily connections, as exemplified by the emergence of 5G and the forthcoming 6G networks. Radio Resource Management (RRM), a major challenge for wireless communication systems, aims to optimize the utilization of limited wireless resources (such as spectrum and power) to enhance network performance and satisfy user demands. Nevertheless, traditional approaches to wireless resource management are somewhat constrained in large-scale, complex, and dynamically changing network environments. These approaches have long formally developed design goals and constraints as optimization problems, which are subsequently solved to some extent using mathematical techniques. Despite the effectiveness of these methods in most network environments, they may not be optimal in complex, dynamically changing network environments. In contrast, machine learning has seen significant advancements in terms of adaptability, robustness, scalability, and predictive ability, particularly in the context of 6G [1][2]. The integration of wireless communication networks and machine learning is regarded as a promising avenue for future development. The existing research has focused on specific aspects of deep learning in resource management, but lacks a comprehensive overview of developments over recent years.
This paper examines three algorithms that are currently being utilized in the domain of wireless resource management, outlines the fundamental principles of each algorithm, and explores the advantages, research progress, and future challenges of each algorithm, which aims to demonstrate a comprehensive understanding of how these algorithms can optimize wireless resource management. Through a comprehensive account of existing research and practical applications, this paper provides a valuable reference for further research on the application of machine learning in resource management for wireless communications. Therefore, the paper provide insights into the capabilities and limitations of various algorithms so as to promote research progress and practical applications in this area. In short, it will facilitate the development of more efficient and adaptive network solutions.
2. Overview of Machine Learning Methods
2.1. Graph Neural Networks
Graph neural networks, a class of neural networks specialized in processing graph-structured data, such as social networks, and knowledge graphs, achieve information transfer through the product of the pro-order matrix A and the graph signal X. And the core idea is to take advantage of the relationship between nodes and edges in the graph structure, and to better capture the global and local features of the graph through successive iterative updating of each node, where each node receives the information generated by the neighboring nodes information from neighboring nodes and update itself with it. Mathematically, the convolution operation of GNN can be expressed as (where D means the degree matrix of A, and W represents the learnable weight matrix):
Z=F(A,X)=D−1A X W (1)
GNNs are currently widely utilized in a variety of domains, including knowledge retrieval, molecular chemistry, traffic networks, and wireless resource allocation. Nevertheless, despite the obvious advantages of graph neural networks in the processing of graphs, and the good scalability of some irregular data structures, there are still some problems. Specifically, due to the high computational complexity, requires a long computation time and more arithmetic support; secondly, after many runs, graph neural networks also suffer from the problem of information loss.
2.2. Deep Reinforcement Learning
Deep reinforcement learning, a machine learning algorithm, combines two techniques, deep learning and reinforcement learning, where deep neural networks can handle many complex data structures, while reinforcement learning can interact with the environment and learn strategies to maximize long-term returns. Integrating the features of both, deep reinforcement learning is able to handle high-dimensional states and spatial actions, which allows it to excel in complex environments. The fundamental tenet of deep reinforcement learning algorithms is to enhance an agent's decision-making process through approximation and value-based strategies, encompassing value-based methods and policy gradient methods. Deep Reinforcement Learning is widely used in game AI, robot control, automatic driving, etc., and has been used in a mature way. In wireless resource management, deep reinforcement learning has also been applied in many aspects, such as spectrum allocation, power control and load balancing, and has better performance than traditional algorithms in these aspects.
2.3. Federated Learning
Federated learning, a novel cryp-tographic distributed machine learning algorithm that has emerged in recent years, which allows multiple participants to jointly train a global model of artificial intelligence without the data leaving the local area. The algorithm works by initializing a global model at a central server, then dividing the training tasks of the model locally and training them, uploading the updated parameters to the central server after the training is completed, and finally employing an aggregation algorithm to integrate these parameters into a new global model. By iterating the above steps, the final model converges [3]. In recent years, federated learning has been widely used in various domains such as smart cities, smart healthcare, smart transportation, smart grids and social media because of its data privacy protection, less bandwidth requirement and better model performance [4][5].
Federated learning also has some drawbacks in practical applications, for example, the data of the participants may not be independently and identically distributed (Non-IID), which affects the stability of the model training and the convergence speed, the system needs more complex mechanisms to manage multiple participants, and the frequent transmission of the model parameters brings more communication overhead, which need to be improved by future research.
3. Applications of Machine Learning in Resource Allocation
3.1. Graph Neural Networks in Resource Allocation
The power control is of paramount importance in all wireless communication systems, given the intricate relationships and interactions between a multitude of nodes. Traditional approaches are typically founded upon convex optimization methods. However, a considerable number of issues are non-convex, and the application of fixed-parameter algorithms to identify the optimal control method and scale crosswise is often challenging. In contrast, GNNs are capable of capturing the potential non-linearities and complex dynamics within the network with greater efficiency, as they represent the nodes and their connectivity relationships as graph structures
In power control, GNNs outperform other algorithms on specific problems. Based on the algorithmic alignment framework [6], it is demonstrated that GNNs outperform multilayer perceptrons (MLPs) for the K-user interference management problem, and that the performance gap between these two methods increases with K [7]. In the field of radio resource management, GNNs also demonstrate efficacy. Yifei Shen et al. verified the equivalence between MPGNN and a series of distributed optimization algorithms, and accordingly analyzed the performance and generalization ability of MPGNN-based methods. Extensive simulation results using power control and beamforming as examples show that the proposed method, trained in an unsupervised manner using unlabelled samples, matches and even outperforms classical optimization algorithms without domain-specific knowledge in some cases. In addition, the proposed method is highly scalable, which takes only 6 ms to solve the beamforming problem in an interfering channel with 1000 transceiver pairs on a single GPU. This shows that MPGNNs not only possess the advantages of high efficiency and scalability, but also provide theoretical reliability and interpretability in solving large-scale wireless resource management problems [8].
3.2. Deep Reinforcement Learning in Resource Allocation
In recent times, the field of physical layer design and resource allocation has witnessed a surge in interest with regard to interference management in multicellular networks. The deployment of a large number of base stations (BSs) covering an area of point-to-point wireless connections results in the formation of multi-cell interference systems [9]. Traditionally, numerical optimization methods have been the primary means of solving resource allocation problems [10], including Weighted Minimum Mean Square Error, Fractional Planning, and Interference Pricing. Typically, these methods employ key performance metrics, such as channel realizations, as inputs through iterative algorithms and output an optimal power allocation strategy. However, most of these optimization problems are challenging to resolve due to their non-convexity and high-dimensional parameters.
At present, deep reinforcement learning techniques have been employed to address wireless resource management challenges, demonstrating significant advantages in scenarios where traditional methods are inadequate. n addition, the method has been applied to a variety of areas such as spectrum access, throughput maximization, power and channel allocation, thereby enhancing the performance of 5G wireless networks. The two basic DRL algorithms, Q-learning and DQL, represent policy-based and value-based design techniques, respectively. The classical Q-learning technique has been used for downlink resource allocation in Non-Orthogonal Multiple Access (NOMA) networks [11], while DQL has been used to solve problems that cannot be handled in traditional learning systems. For example, in Cloud RAN, Xu Z et al. used DQL for power allocation to reduce power consumption and ensure per-user reliability constraints [12]. Besides, Li X et al. applied DQL to a distributed spectrum sharing scheme to facilitate resource allocation to multiple users in a non-cooperative manner [12].The experimental results show that with, the help of learned power control policies, an auxiliary user is capable of intelligently adjusting its transmit power so as to reach the target state from any initial state within a few transition steps.
3.3. Federated Learning in Resource Allocation
In wireless resource management, an efficient resource management and device scheduling scheme is of paramount importance, which enables other devices to participate in the learning process, thus improving the overall learning performance. Existing federated learning algorithms primarily concentrate on resource optimisation [13], through which the communication burden of joint learning in wireless networks can be reduced, based on which, Zhixiong Chen et al. proposed Partial Model Aggregation-FL (PMA-FL), designed to reduce the amount of data in the transmission phase and to improve the learning performance in heterogeneous local data distribution scenarios. Experimental results show that the proposed FL algorithm achieves faster convergence and higher model accuracy than the benchmark scheme, with accuracy improvements of 3.13% and 11.8% on the MNIST and CIFAR-10 datasets, respectively. Furthermore, on the MNIST dataset, the proposed joint equipment scheduling and resource management algorithm reduces the energy budget by approximately 29% and the time budget by 20%, while achieving higher accuracy than the benchmark. On the CIFAR-10 dataset, the proposed algorithm achieves slightly higher accuracy than the benchmark solution and reduces the energy budget by 25% or the time budget by 12.5% [14].
4. Future Outlook and Recommendations
In the future, machine learning algorithms will play an increasingly significant role in the allocation of wireless communication resources. As 6G communications evolve, machine learning will provide comprehensive assistance in a range of areas, including network orchestration and management, physical layer signal processing, data mining, and service-oriented context-aware communications. In particular, the development of deep learning techniques will significantly enhance the performance and reasoning speed of communication systems through the application of deeply unfolded signal processing methods. In this context, Multi-Intelligent Deep Reinforcement Learning (MADRL) will become an effective tool to cope with dynamic resource allocation in complex wireless network environments. MADRL enables the system to perform efficient resource allocation and management in response to changing network conditions and user demands. Besides, edge intelligence will assume a pivotal role in future 6G networks, facilitating the deployment of large-scale IoT device-dense networks and markedly enhancing the flexibility and efficiency of resource allocation [15].
To facilitate advancement in this field, interdisciplinary research will become a crucial undertaking. The combination of knowledge from multiple disciplines, including machine learning, signal processing, and communication engineering, can facilitate the development of more advanced and efficient resource allocation algorithms. Furthermore, the scalability and efficiency of the algorithms must be a primary focus to meet the demand for large-scale and high-density deployment in future communication networks. In addition, ensuring the security and interoperability of algorithms in resource allocation represents an important direction for future research.
5. Conclusion
This paper presents a summary of the application of deep reinforcement learning, federated learning and graph neural networks in wireless resource management. A review of the current research findings indicate that these techniques show great potential in improving the performance of wireless networks and solving complex resource management problems. Specifically, the deep reinforcement learning technique excels in spectrum access, throughput maximization, power and channel allocation, which is able to effectively deal with non-convex optimization problems that are difficult to be solved by traditional methods. The federated learning technique achieves the objective of co-training a global model with multiple participants while preserving data privacy, thus reducing bandwidth requirements and enhancing model performance. The graph neural networks demonstrate their benefits in resource allocation and optimization problems when confronted with complex network structure data. However, these algorithms still exhibit certain limitations. Future algorithms should enhance their environmental adaptability and multimodal multi-task learning ability, and combine with traditional algorithms to develop more sophisticated models. Through continuous research and innovation in these directions, the application of machine learning algorithms in wireless resource management will become more efficient, intelligent, and reliable, thus promoting the development of future wireless communication networks.
References
[1]. Du, J., Jiang, C., Wang, J., Ren Y. and Debbah, M. (2020) Machine Learning for 6G Wireless Networks: Carrying Forward Enhanced Bandwidth, Massive Access, and Ultrareliable /Low-Latency Service, IEEE Vehicular Technology Magazine, 15(4): pp. 122-134
[2]. Rekkas, V.P., Sotiroudis, S., Sarigiannidis, P., Wan, S., Karagiannidis, G.K. and Goudos, S.K. (2021) Machine Learning in Beyond 5G/6G Networks - State-of-the-Art and Future Trends. Electronics 2021(10): 2786.
[3]. Nikolaidis, F., Symeonides, M. and Trihinas, D. (2023) Towards Efficient Resource Allocation for Federated Learning in Virtualized Managed Environments. Future Internet 2023, 15, 261.
[4]. Gadekallu, T.R., Pham, Q.V., Huynh-The, T., Bhattacharya, S., Maddikunta, P.K.R., Liyanage, M. (2021) Federated Learning for Big Data: A Survey on Opportunities, Applications, and Future Directions. arXiv 2021, arXiv:2110.04160.
[5]. Li, T., Sahu, A.K., Talwalkar, A. and Smith, V. (2020) Federated Learning: Challenges, Methods, and Future Directions, IEEE Signal Processing Magazine, 37(3): 50-60
[6]. Xu, K., Li, J., Zhang, M., Du, S., Kawarabayashi, K. and Jegelka, S. (2020) What can neural networks reason about?, in Proc. Int. Conf. Learning Representations.
[7]. Shen, Y., Zhang, J., Song, S.H. and Letaief, K.B. (2021) AI Empowered Resource Management for Future Wireless Networks, 2021 IEEE International Mediterranean Conference on Communications and Networking (MeditCom), Athens, Greece, 2021, pp. 252-257.
[8]. Shen, Y., Shi, Y., Zhang, J. and Letaief, K.B. (2021) Graph Neural Networks for Scalable Radio Resource Management: Architecture Design and Theoretical Analysis, IEEE Journal on Selected Areas in Communications, 39(1): pp. 101-115.
[9]. Gesbert, D., Hanly, S., Huang, H., Shitz, S.S., Simeone, O. and Yu, W. (2010) Multi-Cell MIMO Cooperative Networks: A New Look at Interference. IEEE J. Sel. Areas Commun. 2010, 28, 1380-1408.
[10]. Venturino, L., Prasad, N. and Wang, X. (2009) Coordinated Scheduling and Power Allocation in Downlink Multicell OFDMA Networks. IEEE Trans. Veh. Technol. 2009(58): 2835-2848.
[11]. Zhai, Q., Bolić, M., Li, Y., Cheng, W. and Liu, C. (2021) A Q-Learning-Based Resource Allocation for Downlink Non-Orthogonal Multiple Access Systems Considering QoS. IEEE Access 2021, 9, 72702-72711.
[12]. Li, X., Fang, J., Cheng, W., Duan, H., Chen, Z. and Li, H. (2018) Intelligent power control for spectrum sharing in cognitive radios: A deep reinforcement learning approach. IEEE Access 2018, 6, 25463-25473.
[13]. Zeng, Q., Du, Y., Huang, K. and Leung, K.K. (2021) Energy-efficient resource management for federated edge learning with CPU-GPU heterogeneous computing, IEEE Trans. Wireless Commun., 20(12): 7947-7962.
[14]. Chen, Z., Yi, W., Shin, H., Nallanathan, A. and Li, G.Y. (2024) Efficient Wireless Federated Learning with Partial Model Aggregation, IEEE Transactions on Communications (Early Access)
[15]. IEEE Xplore, (2022) How will Machine Learning Redefine Wireless Communication for 6G?,https://innovate.ieee.org/innovation-spotlight/machine-learning-wireless-communication-6g/
Cite this article
Lu,M. (2024). Application of machine learning algorithms in resource allocation for wireless communications. Applied and Computational Engineering,102,37-42.
Data availability
The datasets used and/or analyzed during the current study will be available from the authors upon reasonable request.
Disclaimer/Publisher's Note
The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of EWA Publishing and/or the editor(s). EWA Publishing and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
About volume
Volume title: Proceedings of the 2nd International Conference on Machine Learning and Automation
© 2024 by the author(s). Licensee EWA Publishing, Oxford, UK. This article is an open access article distributed under the terms and
conditions of the Creative Commons Attribution (CC BY) license. Authors who
publish this series agree to the following terms:
1. Authors retain copyright and grant the series right of first publication with the work simultaneously licensed under a Creative Commons
Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this
series.
2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the series's published
version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial
publication in this series.
3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and
during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See
Open access policy for details).
References
[1]. Du, J., Jiang, C., Wang, J., Ren Y. and Debbah, M. (2020) Machine Learning for 6G Wireless Networks: Carrying Forward Enhanced Bandwidth, Massive Access, and Ultrareliable /Low-Latency Service, IEEE Vehicular Technology Magazine, 15(4): pp. 122-134
[2]. Rekkas, V.P., Sotiroudis, S., Sarigiannidis, P., Wan, S., Karagiannidis, G.K. and Goudos, S.K. (2021) Machine Learning in Beyond 5G/6G Networks - State-of-the-Art and Future Trends. Electronics 2021(10): 2786.
[3]. Nikolaidis, F., Symeonides, M. and Trihinas, D. (2023) Towards Efficient Resource Allocation for Federated Learning in Virtualized Managed Environments. Future Internet 2023, 15, 261.
[4]. Gadekallu, T.R., Pham, Q.V., Huynh-The, T., Bhattacharya, S., Maddikunta, P.K.R., Liyanage, M. (2021) Federated Learning for Big Data: A Survey on Opportunities, Applications, and Future Directions. arXiv 2021, arXiv:2110.04160.
[5]. Li, T., Sahu, A.K., Talwalkar, A. and Smith, V. (2020) Federated Learning: Challenges, Methods, and Future Directions, IEEE Signal Processing Magazine, 37(3): 50-60
[6]. Xu, K., Li, J., Zhang, M., Du, S., Kawarabayashi, K. and Jegelka, S. (2020) What can neural networks reason about?, in Proc. Int. Conf. Learning Representations.
[7]. Shen, Y., Zhang, J., Song, S.H. and Letaief, K.B. (2021) AI Empowered Resource Management for Future Wireless Networks, 2021 IEEE International Mediterranean Conference on Communications and Networking (MeditCom), Athens, Greece, 2021, pp. 252-257.
[8]. Shen, Y., Shi, Y., Zhang, J. and Letaief, K.B. (2021) Graph Neural Networks for Scalable Radio Resource Management: Architecture Design and Theoretical Analysis, IEEE Journal on Selected Areas in Communications, 39(1): pp. 101-115.
[9]. Gesbert, D., Hanly, S., Huang, H., Shitz, S.S., Simeone, O. and Yu, W. (2010) Multi-Cell MIMO Cooperative Networks: A New Look at Interference. IEEE J. Sel. Areas Commun. 2010, 28, 1380-1408.
[10]. Venturino, L., Prasad, N. and Wang, X. (2009) Coordinated Scheduling and Power Allocation in Downlink Multicell OFDMA Networks. IEEE Trans. Veh. Technol. 2009(58): 2835-2848.
[11]. Zhai, Q., Bolić, M., Li, Y., Cheng, W. and Liu, C. (2021) A Q-Learning-Based Resource Allocation for Downlink Non-Orthogonal Multiple Access Systems Considering QoS. IEEE Access 2021, 9, 72702-72711.
[12]. Li, X., Fang, J., Cheng, W., Duan, H., Chen, Z. and Li, H. (2018) Intelligent power control for spectrum sharing in cognitive radios: A deep reinforcement learning approach. IEEE Access 2018, 6, 25463-25473.
[13]. Zeng, Q., Du, Y., Huang, K. and Leung, K.K. (2021) Energy-efficient resource management for federated edge learning with CPU-GPU heterogeneous computing, IEEE Trans. Wireless Commun., 20(12): 7947-7962.
[14]. Chen, Z., Yi, W., Shin, H., Nallanathan, A. and Li, G.Y. (2024) Efficient Wireless Federated Learning with Partial Model Aggregation, IEEE Transactions on Communications (Early Access)
[15]. IEEE Xplore, (2022) How will Machine Learning Redefine Wireless Communication for 6G?,https://innovate.ieee.org/innovation-spotlight/machine-learning-wireless-communication-6g/