1. Introduction

In recent years, as the demand for air traffic has increased, air traffic systems in all countries have been under enormous pressure, causing significant air traffic delays. Based on statistics from the CAAC, during the period of 2015-2019, the average normal rate of passenger flights nationwide was 75.71%, with an average delay time of 18 min, of which the normal rate of passenger flights in 2015 was only 68.33%, with an average delay time of over 20 min. According to the statistics of the US Bureau of Transportation Statistics [1-2]. During the same period, the on-time performance of major US airlines was only 79.99%. Although there was a downward trend in global air transport due to the impact of the new crown epidemic, delays fell briefly. However, with the recovery from the epidemic, the domestic and international aviation industry is gradually picking up and demand for air traffic is still experiencing strong growth. While flight delays affect passengers' journeys, they also cause huge losses to the national economy. According to statistics, flight delays indirectly caused an economic loss of about 350 billion yuan to China in 2013 [3]; the economic loss caused by flight delays in the United States in 2007 was about US$3.3 billion [4]. To address the above issues, there is an urgent necessity to improve air traffic management capabilities and to deploy air traffic flow management strategies in advance to reduce losses. Currently, relevant regulatory bodies and researchers have started to promote computer modelling as an alternative research method. Among them, machine learning, a computer modelling approach that has been developing rapidly in recent years, has been widely used in many fields such as natural language processing, image recognition and autonomous driving [5-7]. The learning model of machine learning is similar to that of humans, in that a program learns generic patterns from a large amount of data, builds a model and attempts to predict the target value, and by comparing the difference between predicted and true values, continues to trial and error, iterates and optimises, and eventually outputs a model with high accuracy. Machine learning has a powerful learning capability and versatility that could open up new opportunities for air traffic management.

This paper therefore specifically explores the current application of machine learning in air traffic management research. The first step introduces how machine learning can be applied to air traffic management in terms of basic machine learning concepts and machine learning modelling processes (data set construction, data pre-processing, model selection and construction, model analysis) respectively; the second step discusses existing research in terms of the three components of air traffic management (air traffic services, air traffic flow management, airspace management) respectively; on the basis of the present situation of research in the third step, the challenges and prospects of the current research on machine learning in air traffic management are pointed out. Finally, it is concluded that, despite the challenges, the introduction of machine learning has significant implications for the research and development of air traffic management, and this thesis will provide future researchers with references on research directions to facilitate the application of machine learning in real production environments.

2. Research methods for machine learning in air traffic management

2.1. Machine learning

Depending on the learning approach, machine learning for air traffic control can be loosely categorised into supervised learning, unsupervised learning, semi-supervised learning, and reinforcement learning. In the field of air traffic management, supervised learning is generally used for the prediction and validation of air traffic metrics with explicit input and output values, such as for predicting air traffic flow parameters; unsupervised learning is generally used for the exploration of unknown patterns with non-targeted outputs, such as for helping researchers assess the complexity of airspace; semi-supervised learning is generally used for processing partially labelled training data, usually a large amount of un Semi-supervised learning is generally used to process partially labelled training data, usually a large volume of unlabelled data and a small volume of labelled data, such as automatic speech recognition systems in air traffic control; and reinforcement learning is generally used to solve the problem of maximising the reward or achieving a specific goal by learning strategies during the interaction of an intelligence with the environment, and differs from the first three mainly in the reinforcement signal, which is provided by the environment as an evaluation of the resulting action, which is generally used to provide decision support to air traffic controllers through autonomous learning.

According to the theory underlying machine learning algorithms, machine learning can be classified into traditional machine learning algorithms and neural network algorithms.Traditional machine learning algorithms such as multiple linear regression, Bayesian[8], decision trees[9], support vector machines[10], and a range of integrated learning algorithms such as random forests[11], XGBoost[12], etc. are now widely used in various fields including air traffic prediction. The Scikit-learn machine learning library in Python has integrated all of the above traditional machine learning algorithms for use by researchers[13]. Neural network algorithms process information by simulating the structure of neurons in the human brain, consisting of multiple layers of neurons, each of which receives signals from the previous layer of neurons, performs weighting operations, generates an activation response through a non-linear activation function (e.g., Sigmoid function) and passes the output signal to the next layer of neurons. With the rapid growth in computing power of today's computers, increasingly complex neural networks, i.e., deep neural networks[14], have emerged and shown great potential for application. The main types of deep neural networks in common use today are convolutional neural networks[15], recurrent neural networks and fully connected neural networks[16]. Frameworks for using neural networks such as Pyroch, Tensoflow and Caffe have been open sourced to facilitate researchers' use[17-19].

2.2. Machine learning model building process

Understanding the process of how machine learning can be adapted to the air traffic management field can effectively help researchers build intuition. The process of using machine learning in the area of air traffic management is shown in Figure 1.

Figure 1. Workflow of the application of machine learning in air traffic management.

2.2.1. Dataset construction. A large, high-quality dataset is the basis for building a reliable machine-learning model, and the construction of a dataset is generally divided into a number of steps. The first step is data collection, which is generally derived from published papers, relevant public datasets and some internal datasets of relevant regulatory agencies or departments. In the case of severe data deficiencies, new datasets can also be constructed from ADS-B messaging systems and ATC communication records for machine learning mapping purposes. Then comes the extraction of experimentally relevant information as model inputs, such as aircraft altitude, heading, speed and position, etc. Finally, the objectives of the study (future air traffic flow, evaluation indicators, etc.) are taken as the predicted outputs of the model.

Here are some common air traffic databases.

Table 1. Popular air traffic databases.

2.2.2. Data pre-processing. The collected data will have problems such as missing data and irregular data format, so in order to improve the quality of the data, data cleaning and pre-processing are needed before modelling. When a feature in the data set has a large number of missing values, the feature will generally be discarded, otherwise, it will have a detrimental impact on the model effect. In the case of outliers in the data, the data can usually be removed directly. In addition, the data is usually normalised or normalised in order to limit the values to a certain range and to prevent the model from being biased due to the large value of a feature. It is important to note that string variables that cannot be recognised by machine learning (e.g., categorical variables) can be converted to numeric variables that can be brought into the model using one-hot encoding.

2.2.3. Model selection and construction. Machine learning models, such as classification and regression predictions, are primarily utilized for the prediction of discrete and continuous data. Most common algorithms such as decision trees, random forests, support vector machines, artificial neural networks and XGBoost can be used for both classification and regression, except for plain Bayesian and k-neighbourhood algorithms which are only used for classification, and multiple linear regression which can only be used for regression prediction. Within the air traffic management domain, classification models can be used to determine whether current air traffic conditions will cause delays, while regression models are often used to predict air traffic flows in the airspace over time.

Model building is a central part of the machine learning process in air traffic management. Before building the model, the pre-processed data set is split into a training set, a validation set and a test set, usually in the ratio of 7:2:1, and combined with appropriate algorithms, the training set is applied to construct the model and the validation set is brought into the model to test the predictions and optimise the hyperparameters of the model. The hyperparameters of the model (e.g., the number of decision trees in the random forest, the maximum depth of the tree, the number of neurons in each layer, etc.) often have a significant impact on the prediction performance of the model. When using the Python language for machine learning, the GridSearchCV package in the Scikit-Learn library is often used to grid search the hyperparameters of the model to find the best model hyperparameters, and finally the test set is applied to check the model's ability to generalise. Alternatively, the robustness of the model can be tested using k-fold cross-validation by dividing the dataset into k copies, where k-1 copies of the data are used to train the model, and then a separate copy is used to validate the model. k times of cross-validation are repeated, and the average k times of validation results are used as the performance output of the model. 10-fold cross-validation is currently predominantly used.

Depending on the type of target value of the data, these models can be grouped into regression models and classification models. Regression models are used to predict data with continuous numerical variables as the target value, and classification models are used when the target value is a discrete type of variable. Regression models are generally evaluated by the coefficient of resolvability ( \( {R^{2}} \) ) and root mean square error ( \( RSME \) ), while classification models are evaluated by the accuracy, precision, recall and receiver operating characteristic curve ( \( ROC \) ) and area under the curve of \( ROC \) ( \( AUC \) ) [20].

2.2.4. Model analysis. Because the process of building machine learning models is automated and opaque, they are also often referred to as 'black box' models [21], which makes it difficult for researchers to obtain relationships between results and variables from model predictions. Therefore, model interpretability is also one of the key areas of interest for researchers. Also, methods commonly used to explore model interpretability such as sensitivity analysis and random forest ranking of feature importance can help researchers find important features in the data.

3. Related research in machine learning in air traffic management

The International Civil Aviation Organization (ICAO) Annex on Air Traffic Services defines air traffic as: Every aeroplane in flight or operating in the apron's manoeuvring area. And in this context, the manoeuvring area is defined as that area of the airport where aircraft are used for take-off, landing and taxiing, excluding the apron.Air traffic management is defined in ICAO Doc 4444 PANS-ATM as: Safe, economical and effective regulation of air traffic and airspace ( covering air traffic services, airspace management and air traffic flow management) is achieved by working in partnership with all parties to offer facilities and seamless services that include air and ground functions. The system of air traffic management could therefore be represented by the diagram below.

Figure 2. ATM system structure.

By the above definitions, the existing studies are discussed in three categories.

3.1. Research on machine learning in air traffic services

Air Traffic Services (ATS), usually aimed at guaranteeing the safety and order of the flow of traffic (Air Traffic Control (ATC) services), and providing information to crew members (Flight Information Services, FIS) and to specific agencies during emergencies (Alert Services).Since ATS is for the safety and orderliness of air traffic flow, it is inevitable to obtain aircraft data in real-time and predict the future trajectory of the aircraft through the ADS-B system, allowing air traffic controllers to judge the future airspace conditions and make timely and reasonable controls and adjustments. Naessens and others have presented a deep learning-based model which is capable to make horizontal trajectory predictions based on historical data of this flight and has been integrated into the air traffic flow and capacity management system of the EUROCONTROL Area Control Centre[22]. However, the model is only suitable for forecasting during the layout phase, not for predicting trajectories over time in the future, and it does not support trajectory prediction within the vertical profile. Shi et al. subsequently introduced an LSTM-based forecasting model that uses the interaction of repeated modules in the LSTM to combine long-term dependencies in the data with a forecasting task to achieve a prediction for a future period, and adds the altitude of the aircraft to the data to complete a forecast of the 4-D trajectory of the flight[23]. Further, Ma presented a CNN-LSTM model by merging the features of CNN to extract spatial features and LSTM to extract temporal features, which greatly improved the prediction accuracy[24].

Air traffic controllers have almost as much responsibility in their work as civil aviation pilots, and the decisions made by air traffic controllers determine the airspace traffic conditions for some time in the coming period, so improving the correctness and rationality of air traffic controllers' decisions is a key point. For this purpose, Yang et al. constructed an ATC raw speech corpus containing both Mandarin and English, collected in a real system and fully annotated and transcribed, which has contributed greatly to subsequent research related to automatic speech recognition in ATC[25]. Helmke proposes a supervised learning algorithm that can automatically learn from the above-mentioned tagged ATC speech and form a command extraction model for land-air conversations, which can greatly alleviate the task of recording air traffic control commands[26]. For air traffic controllers, fatigue is the main cause of air traffic accidents. Therefore, Shen proposes to use a stacked sparse autoencoder to extract the shallow features in the manually selected controller's fatigued speech intonation, and then a densely linked convolutional autoencoder to extract the deep features, and finally use a support vector machine classifier to combine the two features to achieve the judgement of whether the controller is in a fatigued state by the controller's speech intonation[27].

Mollinga et al. proposed an air traffic control model using a graph-based deep learning network to control aircraft movements in order to reduce the traffic density in the area and reduce potential conflicts and collisions[28]. Li and Pham used reinforcement learning models to generate agents to detect potential conflicts in the airspace while issuing commands to control the movement of multiple aircraft by shifting heading angles, ascending/descending altitude levels or increasing/decreasing speed to avoid conflicts instead of air traffic controllers[29-30]. These models can be used to inspire air traffic controllers during training or to assist busy air traffic controllers with decision making in high density traffic situations, and in the future could act as a replacement for human controllers.

In addition to these, airports also fall under the scope of air traffic control, also with the aim of identifying and predicting risk precursors and mitigating risks before accidents occur. Herrema uses machine learning and statistical modelling methods to predict taxi launch times, flight times on final approach and airspeed profiles, and to determine abnormal runway occupancy times in order to predict potential risks and accidents[31].

3.2. Research on machine learning in air traffic flow management

Air Traffic Flow Management (ATFM), its main goal is to moderate the flow of aircraft in the most efficient way possible so that it can avoid jams in some control areas. To manage air traffic flow well, the first step is to complete an air traffic flow forecast. Unlike ground traffic forecasting, air traffic has no physical "roads" and aircraft are able to shift through the airspace at altitude levels, so air traffic forecasting is all about space. Gui et al. constructed an aviation big data platform using historical flight ADS-B data to forecast air traffic flow between various cities using an LSTM prediction model, but the model was unable to complete the real-time air traffic flow counting and prediction for a future period for a particular airspace[32]. In order to address these pain points, Ma et al. proposed a new air traffic flow data format that grids the 3D space and transforms the entire air traffic flow data into a 3D traffic flow matrix (TFM), arguing that this data format can further exploit the temporal and spatial correlation in the data and combine the powerful capabilities of CNN and LSTM in spatial and temporal modelling to form an end-to-end prediction model and demonstrates that such a model has a much higher prediction accuracy in short timescales than other models[33]. In greater depth, Malakis et al. recorded the command actions of real air traffic controllers in a terminal radar approach control simulator under various weather conditions and used decision trees to classify scenarios of air traffic, and this research is expected to enhance the stability of various forecasting models in future in datasets that include weather factors[34].

Airspace congestion due to increased air traffic flow is highly likely to lead to large delays in flights, and therefore flight delay determination and time prediction are also worthy research directions. Gui et al. integrated data such as weather conditions, flight schedules and airport information in an ADS-B-based dataset and proposed a random forest-based machine learning model, which solved the overfitting problem arising from previous schemes using LSTM models and had higher accuracy in binary classification[35]. Liu et al. instead used support vector machines to analyse the effects of regional convective weather as well as arrival demands on ground delay schedules, and trained a random forest-based model for predicting the duration of ground delay radicals, which can provide information to passengers at airports and air traffic services[36].

3.3. Research on machine learning in airspace management

Airspace Management (ASM), which aims to manage scarce airspace resources in the most efficient way to satisfy its numerous users, both civilian and military. This service involves the manner in which airspace is assigned to different users ( through routes, regions, flight levels, etc.) as well as the restructuring that takes place to allow air traffic services to be provided. A precise estimate of airspace capacity is critical to a secure, effective and predictable air transport system and can be used as the basis for scenarios to design and optimise airspace structures. Therefore Zhang et al. proposed a decision tree ensemble-based airspace capacity model to continuously adjust and estimate the capacity of the National Airspace System (NAS) in a real-time dynamic environment, a study that is expected to potentially serve as a guide for the selection of aircraft for temporary re-routing and emergency landings[37]. In addition to this, the complexity of the airspace sector has a considerable contribution to make in air traffic management, and the order of magnitude of traffic density in the future open UAV scenario will be much higher than in the existing situation. Xie et al. proposed a CNN-based framework for airspace sector complexity assessment and devised a new data structure, the Multichannel Traffic Image Scene (MTSI), which composes the airspace in a two-dimensional form on a grid and populated with navigation information, enabling more efficient and flexible extraction of deep features to evaluate airspace complexity[38]. While Li et al. considered that the integration cost of such data requiring expert annotation was too high, they propose an unsupervised learning method via a novel loss function which can better address the adverse effects associated with features related to the complexity of the airspace, including dimensional coupling, category imbalance and boundary overlap[39]. It has also been validated in extensive experiments in six regions of southwest Chinese airspace, demonstrating that the model can provide a more objective evaluation to help air traffic controllers understand airspace complexity. In more depth, Ribeiro et al. used reinforcement learning to make the hierarchical design of the airspace more suitable for the flight paths and traffic conditions of aircraft in the current airspace, reducing the number of conflicts and loss of minimum separation, suggesting that this airspace structure is better able to distribute aircraft more evenly, thereby increasing airspace capacity[40].

4. Challenges and prospects for research of machine learning in air traffic management

4.1. Construction of large open datasets and standards

The performance of a machine learning model is strongly influenced by the quantity and quality of the data. When the data in the dataset used for model training is of high volume and quality, the model can produce more reliable and generalised predictions. Therefore, the construction and improvement of relevant databases as well as the opening and sharing of data are also important challenges currently faced. Although some open air traffic information databases are listed above, the number of such databases is still insufficient and the data within them need to be further improved. Therefore, there is a need to create more standard databases that may comprehensively encompass the experimental data needed for various air traffic management studies, such as weather, aircraft parameters, airport structures, airline flight schedules, etc., so that other researchers can model and validate the data for machine learning purposes. In addition, data sharing is also an important challenge, as information sharing and cooperation mechanisms between experimentalists and computationalists need to be established, and experimental data and codes of research work should be made public as much as possible, which will not only facilitate researchers to engage in model validation and save experimental costs but also promote the development and development of machine learning algorithms.

4.2 Involvement of more external features

The main variables currently used in practical studies are still primarily aircraft data, with a small number of studies using airspace structures, weather data, flight plans, air traffic controller instructions and so on.

However, the complexity of real-world air traffic systems is often much higher, for example, the impact of pilot skill levels and operating habits on individual aircraft, the difficulty of flight due to light, terrain and cloudiness of the route, the timeliness of air traffic controllers' instructions, and the high volume of traffic due to special holidays. All of these factors can greatly influence air traffic conditions, but they have rarely been included in these studies. The characterisation of complex environmental systems requires a wide range of parameters and large amounts of data that are difficult to process and model effectively with traditional linear models, and the fast and accurate predictions of machine learning make it a good solution to this problem. Therefore, advancing the research of machine learning in the field of air traffic management is a necessary way to advance the development of accurate and intelligent air traffic management.

5. Conclusion

This paper discusses the development, applications and prospects of machine learning technology in air traffic management. Although machine learning is still a short distance away from real mass production applications, as a key part of the future intelligent transportation system, with the continuous improvement and upgrading of data computing hardware represented by GPUs and the gradual evolution of deep learning algorithms, the key technologies of artificial intelligence will also be developed significantly. There is a growing trend to introduce artificial intelligence into the air traffic management industry, which can effectively control the growth of air traffic demand due to economic development. It is important to improve the resource utilization of airspace, reduce environmental pollution and reduce the occurrence of air traffic accidents by predicting aircraft trajectories, optimizing the airspace structure and enhancing the productivity of air traffic controllers.

Due to the limitations of the author's research level and the length of this thesis, the three sub-scopes of Air Traffic Services (ATS), namely Air Traffic Control (ATC) Services, Flight Information Services (FIS) and Alerting Services, have not been studied separately and in greater depth and compared. In future research, the author will continue to conduct more investigation and analysis in the direction of Machine Learning in Air Traffic Management.