Deep Learning for Text Sentiment Analysis: A Survey

Research Article
Open access

Deep Learning for Text Sentiment Analysis: A Survey

Lujin Huang 1*
  • 1 Jiangsu Ocean University    
  • *corresponding author 18751817157@163.com
Published on 8 November 2024 | https://doi.org/10.54254/2755-2721/104/20241153
ACE Vol.104
ISSN (Print): 2755-273X
ISSN (Online): 2755-2721
ISBN (Print): 978-1-83558-697-6
ISBN (Online): 978-1-83558-698-3

Abstract

With the continuous improvement of many factors such as corpus, computing power, data scale, and laboratory conditions, how to quickly and accurately understand the emotional tendency in the text has become an important issue in many fields. The current state of affairs and the trajectory that deep learning is pursuing in the realm of text emotion analysis hold significant importance. This article first introduces the definition, importance and classification of emotional analysis, including the analysis of emotional polarity, emotional detection and fine-grained emotion. Subsequently, the key technologies of deep learning were discussed and the application of deep learning in natural language processing, especially LSTM, Bert, and their specific applications in emotional analysis within deep learning models. Then reveal the advantages and limitations of each model by comparing the performance of different deep learning models in emotional analysis tasks. Finally, in combination with current deep learning research, this study summarizes the challenges and research trends in the field of text emotional analysis.

Keywords:

Deep learning, text sentiment analysis, LSTM, BERT.

Huang,L. (2024). Deep Learning for Text Sentiment Analysis: A Survey. Applied and Computational Engineering,104,134-138.
Export citation

1. Introduction

The goal of emotional analysis, a prominent domain within natural language processing (NLP), is to automatically recognize and evaluate the positive, negative, and neutral emotional tenors conveyed in textual data. This technology is not limited to simple emotional polar judgment, but also includes more complicated emotional testing, such as anger, joy, sadness, etc. Emotional analysis holds considerable significance as it enables comprehensive examination of extensive textual data derived from social media platforms. This process assists enterprises and organizations in gaining insights into users' emotional inclinations, consumption patterns, and product experiences. Consequently, precise user profiles and marketing strategies can be developed to aid the government and businesses in obtaining real-time understanding of public emotions and opinions. Furthermore, it contributes to advancements in psychology, sociology, and other related fields through research and development efforts.

The typical applications of deep learning in natural language processing include machine translation, text classification, emotional analysis, and Q & A system. Among these applications, emotional analysis stands out as having extensive application scenarios and significant commercial value, making it a hot topic for the application of deep learning technology.

The purpose of this paper is to investigate the application of deep learning in text emotional analysis. Through an examination of the performance of various deep learning models on emotional analysis tasks, it reveals its advantages and limitations, and discusses the possible development direction of the future. The significance of this study is to provide theoretical reference for researchers in the field of emotional analysis, thereby promoting the further development and application of emotional analysis technology.

2. Sentiment Analysis Basics

Sentiment analysis is the process of analyzing, processing, summarizing and reasoning to a subjective text that carries emotional overtones. It primarily makes use of methods like text analysis, natural language processing, and computational linguistics to methodically detect, extract, measure, and examine subjective data and affective states. The most extensively researched task in sentiment analysis is sentiment polarity analysis. It involves the assessment of the viewpoint represented in the sentiment text, which is usually divided into two categories (positive and negative) or three categories (positive, negative and neutral). Emotion type classification is a further refinement of sentiment polarity analysis. From the psychological point of view, it divides the emotions in the text into more specific emotion categories, such as anger, joy, sadness, fear, disgust and surprise. This way of classification often requires more complex models to capture subtle sentiment changes in the text. Fine-grained sentiment analysis is an advanced form of text sentiment analysis. Compared with traditional sentiment analysis, it can achieve more specific emotional categories, and deeply analyze the object and specific aspects of the sentiment[1].

Conventional techniques for sentiment categorization can be broadly classified into two categories: dictionary-based classic analysis techniques and machine learning algorithm-based techniques. The dictionary-based approach divides the text into distinct sections, extracts relevant keywords, and assesses sentiment values primarily through the application of a set of sentiment dictionaries and rules. Lastly, the text's sentiment orientation is derived from the sentiment value. This approach disregards the drawback of sentiment words' context information and instead depends on the sentiment lexicon's quality. The HowNet sentiment dictionary, NTSUSD Simplified Chinese Sentiment Polarity Dictionary, and Dalian University of Technology Sentiment Vocabulary Ontology are among the sentiment dictionaries that are often used in China[2]. The processing challenges brought on by the irregular word lengths can be lessened by the underlying word embedding technology. Deep learning's abstract features not only mimic word relationships to increase processing flexibility, but they can also mitigate the labor-intensive demands of human feature extraction.

3. Overview of Key Technologies in Deep Learning

When discussing the frontier progress in the field of text sentiment analysis, deep learning technology undoubtedly occupies a core position, and its influence and importance are self-evident. Upon conducting a thorough analysis of the technical kernel in this field, it becomes evident that the exceptional performance of deep learning in processing large-scale text data and accurately capturing complex emotional features can be attributed to its meticulously crafted key technical elements: neural network architecture, optimization algorithm, regularization strategy, and activation function. These elements are intertwined and synergistic, which lays a solid foundation for the excellent performance of deep learning models in text sentiment analysis tasks.

Neural network architectures are the cornerstones of deep learning, delineating pathways through which data traverses within the network. Certain jobs are more effectively addressed by alternative designs. Recurrent neural networks (RNNS) and their variants, LSTM and GRU, are particularly suitable for processing sequential data and are able to capture temporal dependencies in the data. Convolutional neural networks (CNNS), for example, excel at processing image data and automatically extract image features through the utilization of convolution and pooling layers. These carefully designed architectures allow neural networks to efficiently learn and process complex data patterns.

Optimization algorithms in deep learning are in charge of modifying a neural network's weights and biases in order to minimize the loss function and enhance the model's prediction performance. The most widely used optimization techniques are Gradient Descent (GD) and its variations, including Adam and Stochastic Gradient Descent (SGD). To achieve this, the gradient of the loss function with respect to the model parameters is computed, and subsequently, the parameters are updated in the direction opposite to that of the gradient. Adam's algorithm is widely regarded as the preferred optimizer for many deep learning models, since it combines the benefits of the momentum technique with RMSprop, resulting in enhanced convergence speed and improved resilience.

One important tool for preventing deep learning models from overfitting is the regularization technique. Overfitting occurs when a model performs too well on the training set, impairing its capacity to generalize to new data. By including a penalty term for the weights in the loss function and encouraging the model to learn smaller weight values, L1 regularization and L2 regularization simplify the model. During the training phase, the Dropout approach arbitrarily removes a portion of the network nodes in order to lessen interdependencies and enhance the model's capacity for generalization. These regularization strategies support deep learning models in maintaining stable performance on challenging tasks.

Activation function is the key element that introduces nonlinearity into neural networks, which enables neural networks to learn and represent complex data patterns[3]. One of the first activation functions, the sigmoid function transfers the input to the interval (0,1) and is frequently applied in the output layer of binary classification problems. However, when the input value is far from the origin, the Sigmoid function is vulnerable to the vanishing gradient problem. The Rectified Linear Unit (ReLU) function solves this problem by a simple threshold operation, where the output equals the input if the input is greater than 0, and is otherwise 0. The ReLU function and its variants such as Leaky ReLU and PReLU have been widely used in deep learning. The vanishing gradient issue must be taken into consideration by the Tanh function, which is a better Sigmoid function that translates the input into the (-1,1) interval and demonstrating superior convergence properties. The learning capacity and functionality of neural networks are significantly impacted by the selection of these activation functions.

4. Applying Deep Learning to Text Sentiment Analysis

4.1. Public Opinion Monitoring and Crisis Management

In today's era of information explosion, public opinion monitoring and crisis management have become an indispensable part. The introduction of text-based deep learning models, especially LSTM (Long Short-Term Memory) networks, has revolutionized this field. With its unique memory unit, LSTM can accurately capture the emotional fluctuations of the public on a specific topic or event for the time series information in a large number of text data such as news, forum posts, microblogs, and so on. It aims to monitor and analyze the public opinion information of hot microblog searches, to gain an understanding of the public attitude and reaction to a topic or event, and monitor the event of negative public opinion in time.

DENG Lei[2] and his team focused on the disappearance of a 3-year-old boy in Xi 'an on May 17, 2022, which quickly attracted wide attention on the Internet and became an example of the effectiveness of online public opinion monitoring systems. In order to analyze the online public opinion of this event, they built a comprehensive monitoring framework. Firstly, 11,248 data closely related to the event were selected from major portal websites by using advanced keyword search technology. Then, the data were meticulously cleaned, segmented, and filtered for stop words to ensure both the quality and accuracy of the dataset. On the basis of data processing, the Word2Vec model is applied to represent the word vector, and the text data is transformed into a vector form that can be understood by the computer. Furthermore, the LSTM deep learning model is used to analyze the sentiment orientation of the vectorized text, and the dataset is successfully divided into three categories: sensitive information, neutral information and non-sensitive information, containing 7,505, 433 and 3,310 data respectively.

However, in the face of a complex and changeable network public opinion environment, a single LSTM model also faces challenges. Firstly, the training process of the LSTM model is complex, and the demand for computing resources and time is high, especially when dealing with large-scale datasets, which may put high requirements on hardware conditions. Secondly, the LSTM model is prone to overfitting during the training process due to its powerful fitting ability, which requires the adoption of regularization, dropout and other technical means to effectively alleviate. In addition, as a black box model, the decision process of the LSTM model is difficult to intuitively explain, which limits its wide application in application scenarios that require high transparency to some extent. Finally, the performance of LSTM models is highly dependent on the quantity and quality of training data, so in practice, a lot of time and effort should be invested in collecting, cleaning, and labeling high-quality datasets to ensure the effectiveness and accuracy of the model.

In this regard, Cheng Zhecheng[4] proposed a Bi-LSTM sentiment analysis method integrating an improved attention mechanism. This method can reduce the influence of the sentiment degree of the hot search topic itself, pay more attention to the user's own comments, analyze the emotional tendency of the user's comments more accurately, and further improve the accuracy and reliability of sentiment analysis. This innovation not only shows the strong potential of the LSTM model in public opinion monitoring, but also points out the direction for its optimization and upgrade in the future.

4.2. Sentiment Analysis of E-commerce Platform

With the rapid expansion of the Internet, online shopping has become a necessity for the majority of individuals. Consumers leave a large number of product reviews after shopping. In order to fully exploit the value of these text data, the text sentiment analysis technology based on deep learning comes into being. This technology can not only gain insights into users' emotional tendencies and consumption habits, but also provide accurate market insights and decision support for businesses.

Zhao Haobo[5] et al. collected the user comments of a brand mobile phone in Jingdong Mall through the crawler technology, and after a series of preprocessing processes, used the LSTM algorithm to classify the comment text into two categories: positive evaluation and negative evaluation. Experimental results show that LSTM performs significantly better than traditional models such as CNN and SVM in e-commerce comment sentiment analysis, which verifies its unique advantages in processing long text and capturing complex emotional features. This finding not only provides a powerful tool for sentiment analysis of e-commerce platforms, but also provides an important reference for merchants to optimize products and improve services.

However, the accuracy and depth of sentiment analysis still need to be improved. To this end, Cui Teng[6] proposed to construct the BERTBiLSTM-CRF model, which combines the deep language understanding ability of BERT and the contextual information capture ability of BiLSTM, as well as the advantages of CRF in solving long-distance dependency, and realizes the fine extraction and analysis of emotional features in e-commerce reviews. The primary evaluation index is the F1 value, which is derived from the dataset provided by the International Semantic Evaluation Conference. BERT, BILSTM and CNN models were selected for training and results comparison. The BERT-BILSTM-CRF model can obtain better results than the traditional text model in sentiment analysis. This experiment verifies the superiority of the BiLSTM model in text sentiment analysis accuracy, and offers more precise data support for creating user profiles and customizing marketing strategies on e-commerce platforms.

5. Conclusion

The field of sentiment analysis is facing multiple challenges, the first is the surge in demand for cross-language sentiment analysis. In the context of globalization, the differences in sentiment expression between different languages pose a challenge for analysis. Secondly, fine-grained sentiment analysis requires accurate identification of sentiment objects, words and their polarity from the text, which is far beyond the depth and complexity of traditional overall sentiment polarity judgment. Moreover, the emergence of multi-modal sentiment analysis highlights the importance of non-text data such as images, videos, and voice in sentiment expression. How to effectively fuse these multi-modal data has become an urgent problem to be solved.

In the process of coping with these challenges, deep learning technology shows its strong potential and development trend. Firstly, model lightweight has become an important direction of future development, aiming to meet the efficient operation requirements of mobile devices and edge computing scenarios, and reduce computing costs and resource consumption. Secondly, in view of the high cost of obtaining labeled data, unsupervised/semi-supervised learning strategies have gradually attracted attention, which provides a broader data source and learning approach for sentiment analysis. Finally, by designing a more transparent model decision-making process, users 'trust and acceptance of sentiment analysis results can be improved. Improving interpretability may then become crucial in achieving this goal.

In the future, the field of sentiment analysis contains a wealth of potential research directions and technical innovations. Graph neural network-based sentiment analysis techniques are anticipated to greatly increase sentiment analysis accuracy by identifying intricate associations in texts, such as synonyms and antonyms. Concurrently, the sentiment lexicon's dynamic update process, combined with deep learning prediction and manual review, will maintain the timeliness and accuracy of the sentiment lexicon. In addition, the in-depth research on emotion generation technology is expected to promote the construction of dialogue systems with emotional interaction capabilities, and bring more intelligent and humanized interaction experience to users.

The application state and development trend of deep learning technology in text sentiment analysis are examined in this research. To demonstrate the benefits and drawbacks of various deep learning models, a study is undertaken to evaluate their performance in specific sentiment analysis tasks. It is discovered that in sentiment analysis tasks, LSTM and its enhanced deep learning model have demonstrated notable benefits, such as high accuracy and robust generalization capacity. These models, however, also present certain drawbacks, including high processing resource requirements, a heightened risk of overfitting, and insufficient interpretability. Future research areas in the field of sentiment analysis will include improving the interpretability of models, developing unsupervised/semi-supervised learning procedures, and developing lightweight models. Furthermore, it is expected that comprehensive research on graph neural network-based sentiment analysis techniques, dynamic mechanisms for updating sentiment dictionaries, and technologies for generating sentiments will significantly improve the accuracy of sentiment analysis and broaden its potential applications.


References

[1]. Yang Ligong, Zhu Jian, Tang Shiping. A Survey on Text Sentiment Analysis [J]. Journal of Computer Applications,2013,33(06):1574-1578+1607.

[2]. Deng Lei, Sun Peiyang. Network public opinion monitoring system based on deep learning study [J]. Journal of electronic science and technology, 2022, 35 (12) : 97-102. The DOI: 10.16180 / j.carol carroll nki issn1007-7820.2022.12.014.

[3]. Liu Jianwei, Liu Yuan, Luo Xionglin. Research progress of deep learning [J]. Applied Research of Computers,2014,31(07):1921-1930+1942.

[4]. CHENG Z C. Research and implementation of microblog public opinion monitoring model based on deep learning [J]. The computer age, 2023, (11) : 124-126 + 130. DOI: 10.16644 / j.carol carroll nki cn33-1094 / tp. 2023.11.026.

[5]. Zhao Haobo, Tang Fei. Electric business platform based on the deep learning product reviews sentiment analysis [J]. Journal of modern information technology, 2023, 7 (5) : 30-32 + 36. DOI: 10.19850 / j.carol carroll nki. 2096-4706.2023.05.007.

[6]. cui vines. Based on the analysis of deep learning electricity users comment on emotion [J]. Computer knowledge and technology, 2023, 12 (31) : 34 to 37. DOI: 10.14004 / j.carol carroll nki CKT. 2023.1673.


Cite this article

Huang,L. (2024). Deep Learning for Text Sentiment Analysis: A Survey. Applied and Computational Engineering,104,134-138.

Data availability

The datasets used and/or analyzed during the current study will be available from the authors upon reasonable request.

Disclaimer/Publisher's Note

The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of EWA Publishing and/or the editor(s). EWA Publishing and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

About volume

Volume title: Proceedings of the 2nd International Conference on Machine Learning and Automation

ISBN:978-1-83558-697-6(Print) / 978-1-83558-698-3(Online)
Editor:Mustafa ISTANBULLU
Conference website: https://2024.confmla.org/
Conference date: 12 January 2025
Series: Applied and Computational Engineering
Volume number: Vol.104
ISSN:2755-2721(Print) / 2755-273X(Online)

© 2024 by the author(s). Licensee EWA Publishing, Oxford, UK. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license. Authors who publish this series agree to the following terms:
1. Authors retain copyright and grant the series right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this series.
2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the series's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this series.
3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See Open access policy for details).

References

[1]. Yang Ligong, Zhu Jian, Tang Shiping. A Survey on Text Sentiment Analysis [J]. Journal of Computer Applications,2013,33(06):1574-1578+1607.

[2]. Deng Lei, Sun Peiyang. Network public opinion monitoring system based on deep learning study [J]. Journal of electronic science and technology, 2022, 35 (12) : 97-102. The DOI: 10.16180 / j.carol carroll nki issn1007-7820.2022.12.014.

[3]. Liu Jianwei, Liu Yuan, Luo Xionglin. Research progress of deep learning [J]. Applied Research of Computers,2014,31(07):1921-1930+1942.

[4]. CHENG Z C. Research and implementation of microblog public opinion monitoring model based on deep learning [J]. The computer age, 2023, (11) : 124-126 + 130. DOI: 10.16644 / j.carol carroll nki cn33-1094 / tp. 2023.11.026.

[5]. Zhao Haobo, Tang Fei. Electric business platform based on the deep learning product reviews sentiment analysis [J]. Journal of modern information technology, 2023, 7 (5) : 30-32 + 36. DOI: 10.19850 / j.carol carroll nki. 2096-4706.2023.05.007.

[6]. cui vines. Based on the analysis of deep learning electricity users comment on emotion [J]. Computer knowledge and technology, 2023, 12 (31) : 34 to 37. DOI: 10.14004 / j.carol carroll nki CKT. 2023.1673.