Download pdf
The Application of Deep Reinforcement Learning in Stock Trading Models
- 1 University College London
- 2 Shandong University
- 3 South China Normal University
- 4 Capital University of Economics and Business
* Author to whom correspondence should be addressed.
Abstract
Deep Reinforcement Learning (DRL), which integrates the perceptual strength of Deep Learning (DL) with the determination strength of Reinforcement Learning (RL), has emerged as an advanced approach in stock trading. This article focuses on summarizing the research on DRL in stock trading over the past five years, with an emphasis on state definition, action design, reward design, and algorithm selection in stock trading models. Many studies have found that the use of DRL in stock trading can effectively improve investment returns and profitability. In the last years, the adoption of DRL in the stock market has increased and researchers have achieved higher returns and substantial profits through continuous model optimization. However, current research on DRL models faces challenges because of the need for large amounts of complex and uncertain stock market data and the impacts of market volatility and the influence of information asymmetry. This review compares the discrepancies in processing logic among various studies and summarizes the progress made in existing research. It also explores the current challenges and limitations, and discusses potential improvement directions for DRL models in stock trading.
Keywords
deep reinforcement learning, stocks, trading models
[1]. Yang, H., Liu, X. Y., Zhong, S., Walid, A.: Deep reinforcement learning for automated stock trading: An ensemble strategy. In: Proceedings of the first ACM international conference on AI in finance, 1–8 (2020).
[2]. Ghosh, P., Neufeld, A., Sahoo, J. K.: Forecasting directional movements of stock prices for intraday trading using LSTM and random forests.Finance Research Letters 46, 102280 (2022).
[3]. Shi, Y., Li, W., Zhu, L., Guo, K., Cambria, E.: Stock trading rule discovery with double deep Q-network. Applied Soft Computing, 107, 107320 (2021).
[4]. Cao, D. L., Cui, C. R., Yang, X.: A comparative study of domestic financial market investments based on deep reinforcement learning. Journal of Nanjing University (Natural Sciences), 59(2), 333-342 (2023).
[5]. Ma, C., Zhang J., Liu, J., Ji L., Gao F.: A parallel multi-module deep reinforcement learning algorithm for stock trading. Neurocomputing, 449:290-302 (2021).
[6]. Zuo X. D.:Reinforcement learning based on investor sentiment and depth of stock portfolio optimization research. China Management Informationization, 26(3),130-132 (2023).
[7]. Jeong, G., Kim, H. Y.: Improving financial trading decisions using deep Q-learning: Predicting the number of shares, action strategies, and transfer learning. Expert Systems with Applications, (117), 125–138 (2019).
[8]. Jiang, Z., Liang, J.: Cryptocurrency portfolio management with deep reinforcement learning. In: IEEE 2017 Intelligent Systems Conference (IntelliSys), 905-913 (2017).
[9]. Li, Y., Qin, Y.: Deep reinforcement learning for portfolio management. In: Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), 2594-2600 (2017).
[10]. Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., Dieleman, S., Grewe, D., Nham, J., Kalchbrenner, N., Sutskever, I., Lillicrap, T., Leach, M., Kavukcuoglu, K., Graepel, T., & Hassabis, D.: Mastering the game of Go with deep neural networks and tree search. Nature, (529), 484-489 (2016).
[11]. Mohan, V., Singh, J. G., Ongsakul, W.: Sortino ratio based portfolio optimization considering EVs and renewable energy in microgrid power market. IEEE Transactions on Sustainable Energy, 8(1), 219-229 (2016).
[12]. Jiao, Y.: Stock Portfolio Management and Empirical Research Based on Deep Reinforcement Learning. Xian: Northwest University, (2021).
[13]. Zhang, Z., Zohren, S., Roberts, S.: Deep reinforcement learning for trading. arXiv preprint arXiv:1911.10107 (2019).
[14]. Yang, B., Liang, T., Xiong, J., Zhong, C. Deep reinforcement learning based on transformer and U-Net framework for stock trading. Knowledge-Based Systems, 262, 110211 (2023).
[15]. Zhou, B.: Lecture 5 Policy Optimization I, https://github.com/zhoubolei/introRL/blob/master/lecture5.pdf, last accessed 2023/6/8.
[16]. Li, Y, M., Ni, P., Chang, V.: An Empirical Research on the Investment Strategy of Stock Market based on Deep Reinforcement Learning model. In: Proceedings of the 4th International Conference on Complexity, Future Information Systems and Risk (COMPLEXIS 2019), 52-58 (2019). International Conference on Complex Information Systems, Heraklion, Crete, Greece (2019).
[17]. Lele, S., Gangar, K., Daftary, H., Dharkar, D.: Stock Market Trading Agent Using On-Policy Reinforcement Learning Algorithms (2020).
[18]. Luo, S, Y., Lin, X, D., Zheng, Z, X.: A novel CNN-DDPG based AI-trader: Performance and roles in business operations. Transportation Research Part E: Logistics and Transportation Review, 131, 68–79 (2019).
[19]. Nguyen, H, T., Luong, N, H.: Applying Deep Reinforcement Learning in Automated Stock Trading. Soft Computing: Biomedical and Related Applications 981, 280–292 (2021).
[20]. Kang, Q, M., Zhou, H, Z., Kang, Y, F.: An Asynchronous Advantage Actor-Critic Reinforcement Learning Method for Stock Selection and Portfolio Management. In: ICBDR '18: Proceedings of the 2nd International Conference on Big Data Research, pp. 141–145. Association for Computing Machinery, Weihai, China (2018).
Cite this article
Aken,J.;Liang,D.;Lin,Z.;Wang,C. (2023). The Application of Deep Reinforcement Learning in Stock Trading Models. Advances in Economics, Management and Political Sciences,39,215-223.
Data availability
The datasets used and/or analyzed during the current study will be available from the authors upon reasonable request.
Disclaimer/Publisher's Note
The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of EWA Publishing and/or the editor(s). EWA Publishing and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
About volume
Volume title: Proceedings of the 7th International Conference on Economic Management and Green Development
© 2024 by the author(s). Licensee EWA Publishing, Oxford, UK. This article is an open access article distributed under the terms and
conditions of the Creative Commons Attribution (CC BY) license. Authors who
publish this series agree to the following terms:
1. Authors retain copyright and grant the series right of first publication with the work simultaneously licensed under a Creative Commons
Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this
series.
2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the series's published
version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial
publication in this series.
3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and
during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See
Open access policy for details).