Download pdf
A Comparison of LSTM and BERT model for sarcasm prediction
- 1 Southwest Jiaotong University
* Author to whom correspondence should be addressed.
Abstract
Sarcasm prediction is a text analysis task that aims to identify sarcastic and non-sarcastic statements in text. Sarcasm is a figure of speech that uses opposite or contradictory language to express a certain meaning or idea. Sarcasm is usually cryptic, vague, and suggestive, which makes sarcasm prediction a challenging task. In sarcasm prediction projects, techniques of natural language processing are usually leveraged to analyze and classify the text. The main challenge of this task lies in the fact that sarcasm usually has multiple manifestations and needs to consider the contextual and semantic information of the text. The prediction of sarcasm holds significant application value in natural language processing, such as social media analysis, public opinion monitoring, sentiment analysis and so on. In this paper, by controlling variables, the influence of adding the long short-term memory (LSTM) layer and changing the grid structure of the model on the accuracy of prediction results is explored. Moreover, accuracy of the LSTM prediction performance is compared with that of the bidirectional encoder representations from Transformers (BERT) model. At the same time, this paper analyzed and discussed the phenomenon that adding the number of LSTM model layers could not obtain higher prediction accuracy, and the accuracy gap of prediction results between LSTM model and BERT model, and finally obtained relevant conclusions.
Keywords
NLP, sarcasm prediction, LSTM
[1]. Luo, B., Lau, R. Y., Li, C., & Si, Y. W. (2022). A critical review of state‐of‐the‐art chatbot designs and applications. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 12(1), e1434.
[2]. Sarsam, S. M., Al-Samarraie, H., Alzahrani, A. I., & Wright, B. (2020). Sarcasm detection using machine learning algorithms in Twitter: A systematic review. International Journal of Market Research, 62(5), 578-598.
[3]. Janiesch, C., Zschech, P., & Heinrich, K. (2021). Machine learning and deep learning. Electronic Markets, 31(3), 685-695.
[4]. Khanal, S. S., Prasad, P. W. C., Alsadoon, A., & Maag, A. (2020). A systematic review: machine learning based recommendation systems for e-learning. Education and Information Technologies, 25, 2635-2664.
[5]. Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
[6]. Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural computation, 9(8), 1735-1780.
[7]. Wang, S., Wang, X., Wang, S., & Wang, D. (2019). Bi-directional long short-term memory method based on attention mechanism and rolling update for short-term load forecasting. International Journal of Electrical Power & Energy Systems, 109, 470-479.
[8]. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., et, al. (2017). Attention is all you need. Advances in neural information processing systems, 30, 5998-6008.
[9]. Raghu, M., Poole, B., Kleinberg, J., Ganguli, S., & Sohl-Dickstein, J. (2017). On the expressive power of deep neural networks. In international conference on machine learning, 2847-2854.
[10]. Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., & Soricut, R. (2019). Albert: A lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942.
Cite this article
Ding,Y. (2023). A Comparison of LSTM and BERT model for sarcasm prediction. Applied and Computational Engineering,21,63-70.
Data availability
The datasets used and/or analyzed during the current study will be available from the authors upon reasonable request.
Disclaimer/Publisher's Note
The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of EWA Publishing and/or the editor(s). EWA Publishing and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
About volume
Volume title: Proceedings of the 5th International Conference on Computing and Data Science
© 2024 by the author(s). Licensee EWA Publishing, Oxford, UK. This article is an open access article distributed under the terms and
conditions of the Creative Commons Attribution (CC BY) license. Authors who
publish this series agree to the following terms:
1. Authors retain copyright and grant the series right of first publication with the work simultaneously licensed under a Creative Commons
Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this
series.
2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the series's published
version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial
publication in this series.
3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and
during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See
Open access policy for details).