1. Introduction

The financial derivatives market is flourishing globally, and derivative financial instruments are widely used worldwide. Under the premise of increasing instability in the world economy, financial derivatives provide effective tools for stabilizing fluctuations, helping market participants hedge risks such as prices, interest rates, and exchange rates. This has led to an increased demand for using derivative financial instruments for risk management. The use of financial technology for quantitative trading and arbitrage of financial derivatives has also skyrocketed, such as the development of decentralized finance using blockchain technology, which may become the direction of financial market development. However, this has also led to financial crises or other hazards. At the same time, there are certain problems in the development of the derivatives market. The nature of financial derivatives determines their high-risk characteristics: high leverage, significant price fluctuations, liquidity risks, etc. are commonly present in the financial derivatives market. For example, Credit Default Swaps are believed to have caused the 2008 financial crisis; Trader Nick Risen concurrently held trading and clearing responsibilities at the Singapore branch, speculating on the Nikkei index using futures and options. The lack of internal supervision led to risk loss control, resulting in Bahrain Bank losing $1.4 billion and ultimately going bankrupt and being acquired; The "327 treasury bond" futures crisis caused the suspension of China's treasury bond futures market due to the divergence of market expectations on treasury bond discount policy, intense game between the long and short sides, overlapping regulatory gaps and excessive speculation. The existence of these issues indicates the necessity of accurate pricing and more efficient pricing, as well as the urgency of reducing market volatility.

Traditionally, there are various methods for pricing derivatives, such as the Black-Scholes model, Monte Carlo simulation, and finite difference method.

For options, the basic pricing method is the Black Scholes model [1]. The Black Scholes model assumes that the market is frictionless and the underlying asset price follows geometric Brownian motion. Its idea is to buy the option and sell the asset to construct a risk-free pricing combination. The return rate of the risk-free combination is equal to the risk-free interest rate, and the BS partial differential equation is obtained. After solving the partial differential equation, the option price of the European option is obtained; The binary tree model based on discretization time, price change assumption, and risk-neutrality [2]; Monte Carlo simulation based on risk neutral pricing, random path generation, and statistical averaging [3]; The Finite Difference Method, with the core idea of converting continuous differential equations into difference equations through discretization and iteratively solving them on a grid.

The main pricing method for forwards and futures is that a forward contract is an agreement between two parties to buy or sell the underlying asset at a specific price at a future time. Its pricing is based on the holding cost model. Futures are standardized forwards, while for interest rate swaps, the main pricing method is to use the present value of fixed rate and floating rate bonds to be equal, so that the present value of fixed rate bonds is equal to the present value of floating rate bonds.

The traditional derivatives pricing methods have some problems. These pricing methods are based on models, and the classic pricing methods based on Monte Carlo simulation in these models have significant challenges in complex option pricing and risk management due to their large computational complexity, long computation time, and strong dependence on models and parameters; The assumptions of the Black Scholes option pricing model are too ideal, and these assumptions are often difficult to meet in the actual market, which limits the applicability of the model in complex market environments. However, the development of FinTech provides a solution to the issue of option pricing. Among them, deep learning models in machine learning are widely used in derivative pricing. Derivative pricing based on machine learning is based on a large amount of historical data and can handle nonlinear relationships in asset prices, independent of strict assumptions, and can better handle problems with high-dimensional data. These methods do not involve financial theory, but use historical or implicit variables and trading data to estimate option prices in an inductive manner. Although it usually involves some form of parameter formula, at least indirectly, it is not a starting point, but the result of an inductive process. For example, the DGM model can effectively handle high-dimensional problems by solving Partial Differential Equations through neural networks. Time series models such as LSTM and GRU are used to capture temporal dependencies in financial data.

The methods of financial technology are beneficial for understanding the degree of application of machine learning in finance in academia, better optimizing the pricing methods of financial derivatives, improving the accuracy of pricing, correcting traditional derivative pricing methods, enhancing trading efficiency, continuously optimizing trading strategies based on market feedback, and improving trading efficiency. Looking forward to the future trend of derivative product pricing.

This article is mainly divided into three parts: the first part will focus on sorting out the process of machine learning-based derivative pricing methods, and use the paper as an example of practical application; The second part will summarize the specific examples of using machine learning methods to hedge and manage risks through derivatives in the paper; Finally, a summary and outlook on using machine learning for derivative pricing.

2. Basic Derivative Pricing Theory and Machine Learning-based Derivative Pricing Method

2.1. Advantage of Machine Learning in Derivatives Pricing

The rapid development of the financial derivatives market and the increasingly complex product structure pose unprecedented challenges to traditional pricing models. In recent years, some studies have begun to combine neural networks with Monte Carlo valuation, providing new perspectives and methods for pricing complex derivatives. The advantages of machine learning in derivative pricing are gradually emerging, including improving pricing accuracy, achieving automated pricing, reducing operating costs, and adapting to constantly changing market conditions. Especially when dealing with complex derivatives such as basket options with multiple underlying assets, machine learning methods can significantly reduce computational complexity and costs, and improve the efficiency of real-time risk management and arbitrage strategy implementation. In addition, machine learning models can adapt to the dynamic changes in financial markets and provide flexible solutions for pricing derivatives. These studies indicate that machine learning technology has enormous potential and development prospects in the field of financial derivatives pricing.

Scholars have been studying how to apply machine learning to maturity valuation since the 1990s. However, these studies have mainly focused on relatively simple option structures such as vanilla options [4,5], and have not yet studied complex option structures that have no analytical solutions and can only be valued using Monte Carlo methods. Some studies have combined neural networks with Monte Carlo methods for valuation [6], bringing some ideas to related research. Combining deep learning with Monte Carlo simulation ensures both accuracy and speed.

Machine learning has the potential to enhance the precision of pricing mechanisms. Specifically, machine learning models are adept at managing intricate nonlinear relationships, which contributes to improved accuracy in derivative pricing. In their study, Cao and He examined the utilization of machine learning techniques in the pricing of real estate derivatives, highlighting that these methods do not depend on predefined model parameters and can efficiently price real estate derivatives by analyzing data samples. Their comparative analysis of four distinct machine learning approaches revealed that neural network methodologies demonstrate superior accuracy and stability in the pricing of real estate index options [7].

Machine learning methods can achieve automated pricing, reduce the need for manual intervention, and lower operational costs. By directly learning from data, the need for various parameter assumptions about potential market dynamics is eliminated, thereby improving the efficiency of financial markets. Complex derivatives, such as basket options with multiple underlying assets, greatly increase computational complexity and cost due to high-dimensional calculation problems caused by high correlation between assets, which limits the implementation of real-time risk management and arbitrage strategies. High frequency trading requires microsecond-level data processing and decision-making capabilities. During the financial crisis, financial institutions faced significant challenges in assessing the risk exposure of complex derivatives such as CDS or ABS. Risk measurement took a long time, making it extremely difficult to update risk exposure and value assessment in real time during market turbulence. These issues highlight the complex computational challenges that financial institutions face in maintaining competitiveness, managing risks, and meeting regulatory requirements [8,9].

Machine learning models can adapt to constantly changing market conditions and provide flexible solutions for pricing derivatives in today's era of increasing product quantity in derivative financial products. By training neural networks to learn the relationship between model parameters and model prices, the adaptability and flexibility of the model can be improved, thereby better responding to market changes. Heaton et al. discussed how to use neural networks for asset pricing and portfolio management, and proposed a deep learning-based portfolio construction method [10]. Gourieroux and Jasiak proposed using neural networks to model stochastic volatility. By training neural networks, the model can capture the randomness and nonlinear characteristics of volatility, providing more accurate volatility estimates for derivative pricing. It can be seen that machine learning can be personalized in the face of market changes [11].

2.2. Machine Learning-based Derivatives Pricing Process

Machine learning constitutes a subset of artificial intelligence that empowers computer systems to autonomously acquire knowledge and enhance their performance through the utilization of data and algorithms. A range of methodologies within machine learning, including neural networks and random forests, have been employed in the domain of financial derivatives. This discourse will delineate the primary techniques for model training utilizing machine learning approaches for pricing purposes since the 1980s and will elucidate the impact of machine learning on the pricing of derivatives.

2.2.1. Data Preparation and Feature Engineering

Data preprocessing is an important step in machine learning, aimed at converting raw data into a format suitable for model training. The quality of data preprocessing directly affects the performance of the model and the accuracy of the results. Ke and Yang obtained data on option prices and corresponding security prices from Wharton Research Data Services (WRDS), and labeled the data using the average of internal and external prices as the fair value of the options, while excluding nonrepresentative samples for data screening [12]. These efforts to improve data quality and model performance are necessary prerequisites for constructing derivative pricing models based on machine learning.

Feature engineering extracts, selects, and transforms features from raw data to improve the performance and effectiveness of models, with the aim of enabling data to better reflect the essence of the problem. In derivative pricing, target asset prices, execution prices, volatility, and other features are commonly used. In the existing literature, the target price and exercise price are two indispensable variables, and most literature takes the target price S and exercise price K as a features [13]. However, in the execution process, the use of prices in academia can be divided into two types: using the underlying price separately and using the ratio of the underlying price to the execution price. For example, Hutchinson et al. analyzed in their 1994 article that using the ratio instead of the stock price and execution price can reduce the amount of input and make the training of artificial neural networks easier. In recent years, Zheng et al. have used S/K as the input feature in their research [14]. In addition to the target price and strike price, volatility is also widely used as an input feature. The literature that incorporates volatility into input feature engineering includes using historical volatility estimation, using implied volatility as feature engineering, and using GARCH to predict (realized or implied) volatility as feature [13]. In addition, the expiration time τ is also used as an input feature.

The ultimate goal of using machine learning is to price financial derivatives. In the literature, most outputs are derivative prices. Some papers have also studied the ability of artificial neural networks to learn the so-called bias after training, which is the difference between market prices and prices estimated by parameter models. For example, Articles by Boek et al. and Lajbcygier and Connor [15,16]. Because implied volatility can be converted into option prices through the BS formula, some scholars have also used machine learning to output implied volatility.

2.2.2. Common Machine Learning Model Training Methods

Neural Network. Neural networks have shown great potential in the field of financial derivatives pricing due to their powerful nonlinear modeling capabilities and adaptability to high-dimensional data. In order to price and hedge options, Malliaris and Salchenberger, and Hutchinson et al. first used Artificial Neural Networks [17,18]. On a compact set, any continuous function can be around represented by a polynomial, ensuring that artificial neural networks approximate continuous functions in an appropriate manner. Artificial neural networks possess the capability to model nonlinear relationships between input variables and output responses. This characteristic renders them suitable for a variety of applications in the domains of option pricing and hedging. Typically, these networks are designed to learn the pricing of options as a function of several factors, including the underlying asset price, the strike price, and potentially other pertinent features of the options. In the context of pricing tasks, the loss function employed is commonly defined as the squared difference between the actual option price and the price predicted by the artificial neural network [19].

Random Forest. Random forest is a machine learning algorithm based on ensemble learning, which constructs multiple decision trees and integrates their predicted results. In 1984, Breiman et al. invented the Classification Regression Tree (CART) algorithm and used the Gini coefficient for classification [20]. In 2001, Breiman published a paper on random forests [21]. The random forest algorithm can be seen as a set of decision tree clusters, which greatly improves the credibility of the classification results of decision trees through the principle of ensemble learning, and transforms weak classifiers into strong classifiers. Random forests can be applied to modeling implied volatility and path-dependent derivative pricing in derivatives pricing. Vaswani et al. Use random forest regression and artificial neural networks to estimate the price of European option derivatives of NSE Indicators [22].

Support Vector Machine. It is a powerful supervised learning algorithm widely used in classification and regression tasks. The core idea of SVM is to find an optimal hyperplane to separate data points of different categories while maximizing the classification interval, thereby improving the model's generalization ability. Dealing with non-linear and separable data in derivative pricing can help address complex financial market issues. Huang and Wu proposed a hybrid model combining Unscented Kalman Filter and Support Vector Machine for option price prediction, demonstrating the potential of machine learning in the financial field, especially in dealing with nonlinear problems [23].

2.2.3. Evaluation of the Model

The common statistical measurement methods for evaluating the performance of artificial neural networks are Mean Absolute Error, Mean Absolute Percentage Error, and Mean Square Error. Hutchinson et al. introduced Mean Absolute Tracking Error and Prediction Error, which also appeared in many subsequent papers. Buehler et al. introduced the Conditional Value at Risk for evaluating hedging strategies [24]. There are also comparisons between the performance of artificial neural networks and benchmark models, with the most common benchmark being the BS formula. In addition, historical volatility or currency volatility has also appeared in the literature. Blynski and Faseruk compared the historical realized volatility and historical implied volatility of the Bu BS benchmark index [25]. Some scholars have also used Greek letter parameters as benchmarks, such as Ruf and Wang's observation that if a benchmark that includes both delta and Vega hedging is chosen, the performance of artificial neural networks will not even be better than a simple two-factor regression model [19].

3. The use of Machine Learning Techniques in Derivative Pricing

In practice, machine learning based derivative pricing mainly exists as a real-time risk management and control tool for financial institutions, with core application scenarios including arbitrage and hedging, risk measurement and monitoring. By monitoring market data, it is beneficial for relevant institutions to better control financial risks and timely warn and intervene in derivative price fluctuations. This chapter will review the literature on the use of machine learning in specific practices in academia.

3.1. Arbitrage and Hedging

Hutchinson et al. were one of the earliest papers to use artificial neural networks to estimate option prices, and they proposed that derivative pricing and hedging are core issues in financial engineering and risk management [18]. Traditional parametric models, such as the Black-Scholes model, may not be able to fully capture the complex dynamics of the market. A nonparametric approach has been proposed, which uses neural networks to learn and predict reasonable prices and optimal hedging strategies for derivatives, and utilizes the learning ability of neural networks to capture price patterns from historical data rather than relying on specific parameter assumptions. Carverhill and Cheuk were the first scholars to propose a neural network method for a direct output hedging strategy. By analyzing historical data on derivative prices, potential market mispricing and implied volatility were discovered, leading to arbitrage and hedging operations [26]. Buehle et al. proposed a framework for hedging derivative investment portfolios using modern deep reinforcement machine learning methods in the presence of market frictions such as transaction costs, liquidity constraints, or risk limitations. These literatures contribute to achieving asset preservation and appreciation, and improving the return on investment portfolios [24]. In the empirical study conducted by Zhang on option hedging in incomplete markets using deep reinforcement learning, the model results were applied to the financial markets of major countries around the world, and quantitative trading was empirically tested. Finally, it was found that most machine learning models could outperform benchmark models, providing new ideas for quantitative trading of options [27].

3.2. Portfolio Optimization

The use of machine learning models to optimize asset allocation in investment portfolios is an important application in derivatives pricing. Faheem et al. conducted a study on the application of artificial intelligence in the management of investment portfolios, focusing on the comparative performance of various machine learning methodologies. Their research offers improved, adaptable, and accurate strategies for the management of contemporary investment portfolios [28]. The survey by Bartram et al. focused on the most important machine learning methods and empirical results for active portfolio management in the literature [29].

3.3. Risk Measures and Monitoring

Machine learning algorithms learn patterns and patterns from historical data to construct accurate risk assessment models. These models can identify potential risk factors such as market volatility, credit risk, etc., and make accurate predictions. For example, in credit risk assessment, machine learning can improve the accuracy of credit decisions by analyzing borrowers' historical behavior, credit records, and other data to predict their default risk. Khandani et al. discussed the application of machine learning techniques in consumer credit risk assessment and concluded that machine learning models perform well in predicting consumer credit risk, accurately predicting credit card delinquency and default behavior several months in advance [30]. Gu empirically studied the application of machine learning in asset pricing and risk management, solving the problem of measuring asset risk premiums [31].

3.4. The Application of Derivatives Pricing in Quantitative Investment

Quantitative investment is an investment method based on mathematics, statistics, and computer technology, which analyzes market data, formulates investment strategies, and executes trades by constructing models and algorithms. The core of quantitative investment is to use data-driven methods to reduce human emotional interference and improve the scientific and efficient nature of investment decisions. Wang proposed feasible risk management recommendations for structured wealth management products in China at the current stage based on quantitative risk assessment results [32]. Zhang et al. used empirical research to explore two established financial trading strategies and customized them for strategic trading in quantitative markets [33].

4. Conclusion

This article focuses on pricing derivative financial instruments in the context of machine learning. Firstly, introduce traditional parameters and algorithms such as the Black Scholes option pricing model. The assumptions of traditional pricing models are too ideal and have certain limitations when faced with specific situations, resulting in non parametric calculation methods. In recent years, non parametric pricing methods have attracted the attention of researchers, among which machine learning belongs to non parametric pricing methods. The pricing method of machine learning does not rely on a single model but calculates prices from a large amount of historical data. The construction of derivative pricing models based on machine learning generally involves the following processes: data preprocessing, feature engineering, model training, model evaluation, and model application.

The application of machine learning technology in derivative pricing has significant advantages, including improving pricing accuracy, achieving automated pricing, reducing operating costs, and adapting to market changes. Through data preprocessing and feature engineering, machine learning models can be better utilized for pricing derivatives. Common machine learning methods such as neural networks, random forests, and support vector machines have demonstrated strong potential in derivative pricing. In addition, the evaluation methods and application scenarios of machine learning models have also been widely studied and applied.

In the future, the use of machine learning to price derivative financial instruments will become increasingly common, and some of the current problems need to be addressed in the future. The prices of derivatives are influenced by various factors such as price, exchange rate, interest rate, etc., but machine learning still needs better methods for high-dimensional data processing. In irregular time series problems, derivative prices often have irregular time series characteristics, such as the price of futures contracts changing with the expiration date, which poses difficulties for model training and prediction. These problems will be addressed by machine learning in the future. With the development of artificial intelligence, AI will participate in the pricing of derivatives in the future. AI can assist in processing high-dimensional data, adjust pricing strategies in real time according to market changes, and provide assistance in pricing derivatives through machine learning such as deep learning.