1. Introduction

Music therapy is a personalized therapeutic practice led by a trained music therapist, utilizing music elements such as rhythm and harmony to address mental, emotional, physical, social, and cognitive needs [1]. Lyrics discussion is one of the methods used by music therapists that help clients explore emotions, self-expression, and difficult feelings through guided discussions about meaningful song lyrics. One of the therapeutic purposes and objectives of song lyrics discussion is to promote a technique where difficult emotions can be expressed and discussed via the song. [2] This led to the topic of our research on how applications of machine learning, Bert models, are able to support personalized music therapy by classifying songs to different emotions based on the sentiment of the lyrics.

2. Related Work

2.1. Music Classification and Music Therapy

Research in music classification has primarily focused on classifying music by genre. Works such as Poria et al., Elbir et al., Ma, Dhyani1 et al. focused on music classification into genres where diverse methodologies and techniques to improve applicability are discussed [3-6]. Music emotion recognition has also been explored using a variety of approaches with an emphasis on audio feature extraction, as shown in the works of Jamdar et al., Panda et al., and Na and Yong [7-9].

About music therapy, recent research has highlighted the potential of personalized music recommendations based on emotional states. For instance, Modran et al. [10] explored emotion classification and therapeutic effects prediction of music based on individual preferences, mood, and solfeggio frequencies, aiming to assist music therapists in personalized treatment. Shen [11] introduces a particle swarm optimization-based emotion classification model to analyze the impact of interactive music appreciation on college student's mental health. However, these approaches predominantly target monolingual datasets without applications in linguistically diverse contexts.

2.2. Models in Text Classification

Various models have been explored in the task of music emotion recognition. For instance, Agrawal et al. [12] utilize a transformer-based XLNet model to enhance music emotion recognition through lyrics analysis, offering advancements for emotion-based playlist generation and recommendation systems. Among transformer-based models, BERT (Bidirectional Encoder Representations from Transformers) has demonstrated remarkable performance in NLP task text classification. The use of pre-trained language models, as shown by Devlin et al. [13], has significantly reduced the need for extensive labeled datasets. Variants of BERT, such as RoBERTa and DistilBERT, have been fine-tuned for domain-specific tasks, further enhancing their adaptability. In the context of music classification, Pizarro et al. [14] utilized DistilBERT to achieve genre classification, success prediction, and release year estimation. Similar to the purpose of our work, Revathy et al. [15] developed an emotion classification model, LyEmoBert, that analyzes song lyrics to predict emotions—happy, sad, relaxed, or angry. However, these studies often neglect the challenges posed by multilingual and code-switched texts, which are prevalent in global music datasets.

2.3. Contribution of This Work

While prior studies in music classification and emotion recognition have made significant strides, they predominantly focus on monolingual datasets and audio features. Our work addresses the multilingual aspect by classifying emotions from song lyrics across multiple languages using the Bert model, designed to handle the complexities of multilingual datasets, offering a solution that bridges the gap between linguistic diversity and emotion recognition in music. By introducing a multilingual perspective, we are able to expand the potential applications of emotion-aware music recommendation systems and music therapy tools, providing an innovative contribution.

3. Methodology

3.1. Data Collection

To conduct sentiment analysis on Chinese and English song lyrics, we collected relevant data from the following sources:

3.1.1. Chinese Lyrics Sources

Chinese lyrics were gathered from Spotify, ensuring a diverse representation of language and emotional expression within the dataset. A total of 150,000 lyrics were collected, ensuring a wide variety in musical styles and emotional types.

3.1.2. English Lyrics Sources

We used a dataset from Github, which holds over 1000 English song entries with key variables of the artist, genre, title, album, year, and lyrics [16]. For our project, we focused on the key variable lyrics only.

In “SingleLabel” data, as shown in Figure 1, the songs are annotated with emotion labels in 3 categories: sadness, tenderness, and tension. In “OriginalAnnotation” data, as shown in Figure 2, the songs are annotated with Amazement, Calmness, Joyful activation, Nostalgia, Power, Sadness, Solemnity, Tenderness, and Tension.

Figure 1: Single Labeled English Data

Figure 2: Multi Labeled English Data

3.2. Data Preprocessing [1]

Single Labeled English lyrics are tokenized using a BERT tokenizer, which transforms them into token sequences and attention masks, ensuring that each lyric is limited to 512 tokens (max length). Each emotional label (sadness, tenderness, tension) is converted into numerical labels (0, 1, or 2).

For Multi Labeled English lyrics, the emotional labels are binarized (1 for presence of emotion type, 0 for absence) across 9 classes: Amazement, Calmness, Joyful activation, Nostalgia, Power, Sadness, Solemnity, Tenderness, and Tension. A new data frame is created with two columns: the song lyrics (text) and their corresponding binarized labels (labels). The lyrics are tokenized using the DistilBertTokenizer, which converts text into input IDs and attention masks. The maximum token length is set to 128 to ensure uniform input size.

The collected Chinese lyrics underwent several preprocessing steps:

• Data Cleaning: Noise elements such as platform watermarks and advertisements were removed to focus the sentiment analysis on the lyrical content.

• MySQL Storage: The cleaned data (raw lyrics) were stored in a MySQL database for effective management (Figure 3). The storage allowed for easy retrieval, manipulation, and updates. The specific steps included:

1. Storing cleaned lyrics in a MySQL database, using an appropriate schema to save both Chinese and English lyrics along with metadata (e.g., song title, artist, language type).

2. Database fields included: id (song ID), name (song title), singer (artist), lyric (lyrical content), and label (emotion label).

Figure 3: MySQL Storage

• Data Export: Lyrics were exported in the required format for deep learning training using SQL queries. Data was exported in CSV format, with each line containing the lyrical content and corresponding sentiment label (Figure 4).

Figure 4: Data Export

• Tokenization: Lyrics were tokenized using Jieba to enable word-level processing for model training.

• Stopword Removal: Irrelevant words, such as common Chinese particles and English articles, were removed using a stopword list.

3.3. Tools and Models

3.3.1. BERT

Since the Transformer models cannot process raw text directly, we use Bert to convert the text input into numbers, which are serviceable to the models. This process is called tokenization, which will be the response for:

• Splitting the input into words, subwords, or symbols (like punctuation) that are called tokens

• Mapping each token to an integer IDs

• Adding additional inputs that may be useful to the model

BERT receives this list of IDs per sentence and converts them to tensors.

3.3.2. Deep Learning Models

We employed several deep-learning models for sentiment analysis:

• Convolutional Neural Network (CNN): Primarily used for extracting local emotional features from shorter sentences.

• Long Short-Term Memory (LSTM): Excels at processing long textual sequences, capturing emotional flows and long-term dependencies.

• Gated Recurrent Unit (GRU): Similar to LSTM but with a simplified structure for efficient computation, suitable for large lyric datasets.

• GRU + CNN Combination Model: Integrates GRU's sequential modeling capability with CNN's feature extraction to enhance sentiment classification accuracy.

3.3.3. GRU + CNN Embedding

To improve context understanding of lyrics, we introduced the GRU + CNN model. Its bidirectional encoder captures nuanced emotional expressions from the lyrical context, making it particularly effective for multilingual sentiment analysis. The model's pre-trained universal language understanding capability allows it to handle translated lyrics efficiently, preserving emotional information.

3.4. Translation Processing

For handling Chinese lyrics, we utilized the Baidu Translate API, which leverages neural network translation technology for efficient and accurate translations, ensuring the retention of original emotional content. The API call code for translation is shown in Figure 5.

Figure 5: API Call Code for Translation

Figure 6: Flowchart of Sentiment Analysis

The flowchart illustrates the sentiment analysis workflow (Figure 6). Initially, data is processed and cleaned, generating Tensors for model use. Subsequently, models such as CNN, GRU, or LSTM are employed for training, validation, and sentiment classification, ultimately producing inference results.

4. Experiments and Evaluation

4.1. English Lyrics

4.1.1. Single Labeled

We used the pre-trained BERT model that learns contextual relationships between words in the lyrics. After BERT processes the lyrics, the model passes the pooled output (representing the overall sentiment of the lyric) through a dropout layer (prevent overfitting) and a final linear layer to classify the lyrics into one of the emotion classes.

For Training and Evaluation, we utilized the following:

• Optimizer: AdamW optimizer with a learning rate scheduler to adjust the learning rate over time

• Loss Function: Cross-entropy loss is used to measure the difference between the predicted and actual labels

• Training process: For each epoch, feed in lyrics and their corresponding labels, using backpropagation and gradient descent to minimize the loss function

• Validation: evaluate the model on a separate validation dataset after each epoch, tracking accuracy and loss to monitor overfitting and generalization

4.1.2. Multi Labeled data

We used pre-trained DistilBERT as the base to extract features from the lyrics. On top of DistilBERT, a custom classification head is added, consisting of:

• A pre-classifier linear layer

• A dropout layer to prevent overfitting

• A final classifier layer that outputs logits for the nine emotion categories

After the training, we used the Hamming Score to measure the performance.

4.2. Chinese Lyrics

To validate the effectiveness of this methodology, we translated the bilingual lyric dataset and input it into CNN, LSTM, GRU, and GRU + CNN models for training. Model performance was assessed using the following metrics:

• Data Size: Length of the text.

• Accuracy: Proportion of correctly classified results.

Experimental results indicated that the GRU + CNN model outperformed traditional CNN and LSTM models in sentiment classification of translated lyrics, achieving an accuracy of 78.58%. Table 1 shows the comparison of the GRU+CNN model with other models.

Table 1: Comparison [3]

Based on the comparison results in Table 1, we selected the GRU + CNN model as the final predictive model.

5. Results and Discussion

5.1. English Lyrics

As Table 2 and Figure 8 show, the results of the BERT-based sentiment classification model on Single Labeled Dataset show a moderate performance across three sentiment classes: "sadness," "tenderness," and "tension." The model achieved an overall accuracy of 67%, which means about two-thirds of the test samples were correctly classified.

Table 2: Single Labeled English Data

For the metric of the DistilBert model used on Multi Labeled Data, we used the metric hamming score, which showed an overall accuracy of 71%.

Figure 7: Single Labeled English Data

5.2. Chinese Lyrics

Figure 8: Short Sentences in English

Figure 9: Long Sentences in English

Figure 10: Short Sentences in Chinese

As shown in Figures 8-10, the experimental outcomes demonstrate that the inclusion of a translation processing module significantly enhances cross-lingual sentiment analysis accuracy. Particularly in handling complex emotional expressions and semantic transformations, the GRU + CNN model exhibited robust emotional capture capabilities. While CNN and LSTM excel in single-language sentiment analysis, the GRU + CNN model proved more advantageous in multilingual contexts. Moreover, the GRU + CNN model maintained stable performance for both long and short sentences. [3]

6. Conclusion

This research demonstrates the effectiveness of using BERT models for emotion classification in multilingual song lyrics, with a focus on enhancing personalized music therapy practices. The GRU + CNN hybrid model proved to be the most effective in accurately classifying emotions across both English and Chinese datasets, outperforming traditional models like CNN and LSTM, particularly in cross-lingual contexts. These findings suggest that advanced deep learning techniques can significantly contribute to improving sentiment analysis in multilingual settings. Future work could explore expanding the emotion categories, integrating more diverse multilingual datasets, and refining the models further to improve classification accuracy and real-world applicability in therapeutic practices.

Acknowledgement

Yiyang Hong and Yuqi Xue contributed equally to this work and should be considered co-first authors.