1. Introduction

Entering the 21st century, the world is constantly developing towards digitalization and globalization. With the expansion of global trade, a fast and safe currency is urgently needed. Computer scientists around the world have used cryptography, decentralization and other methods to create an encrypted digital currency, the most typical of which is Bitcoin created by Satoshi Nakamoto in 2009. Thanks to the use of decentralized technology, cryptocurrency provides consumers with more choices and privacy [1]. At the same time, it avoids the problem of long processing time due to the commission generated by intermediaries such as banks, reduces transaction costs and improves transaction speed. Despite many advantages, cryptocurrency still has great challenges. The intricate correlation between the internal price fluctuation and trading volume and the external national policies and the emotional changes of the owner makes the price of cryptocurrency fluctuate unpredictably. Considering the great potential for development and broad application scenarios of cryptocurrency in todays and the future, precise control over cryptocurrency prices is crucial.

In the existing research, one method to forecast the price of Bitcoin always used is deep learning. Since deep learning, especially Long Short-Term Memory (LSTM), Bi-directional LSTM (Bi-LSTM) and Gated Recurring Unit (GRU), is excel at capture time series data the strong ability to capture long-term dependencies in time series data notably long-term dependencies, it has been proved that traditional models [2,3]. Zubair et al. proposed a hybrid Bi-LSTM and GRU model to more accurately predict the price of mainstream cryptocurrencies, and used Bidirectional Encoder Representations from Transformers (BERT) and Valence Aware Dictionary and sEntiment Reasoner (VADER) models for sentiment analysis [4]. At the same time, Rathee et al. introduced the Convolutional Neural Network (CNN)-Bi-LSTM integration model to use historical data and real-time API data to predict the prices of Bitcoin, Ethereum, etc. with high accuracy [5]. In order to solve the problem of high volatility and seasonality in cryptocurrency data, Luo et al. Because the currency price in the market is not only affected by internal factors, but also by external user sentiment [6]. Parekh et al. developed the DL model with LSTM and GRU, to conduct emotional analysis from Twitter. Especially when considering the laziness between different currencies, the model can show the improved results [7]. At the same time, Suganda used emotional data in social media such as Twitter to extract emotions and studied the impact of public sentiment on price changes [8]. Some alternative methods show obvious advantages in predicting market trends and assisting investors in portfolio management. Sun et al. improved the accuracy of cryptocurrency forecasting by using integrated models such as decision tree and gradient lifting and Light Gradient Boosting Machine (LightGBM) [9, 10].

Improving the prediction’s accuracy of Bitcoin price is the leading objective in this study. To achieve this, a hybrid deep learning model combining Bi-LSTM and GRU is employed, alongside BERT and VADER for emotion analysis, the FbProphet model, and an attention mechanism. These methods are integrated to address key challenges affecting Bitcoin price prediction, such as public sentiment, seasonality, and issues like gradient vanishing. The attention mechanism is specifically incorporated to allow for dynamic adjustments, further improving the model ability to adapt to changing market conditions. This study aims to provide a more robust framework for predicting Bitcoin prices by leveraging multiple advanced techniques, ultimately enhancing prediction precision.

2. Methodology

2.1. Dataset description and preprocessing

The data set used in this study is from Kaggle [11]. The data set contains the historical transaction data of Bitcoin every minute since January 2012, recording the price and transaction volume of Bitcoin in each time window. Data fields include timestamp, open price, high price, low price, close price, volume, and specific date and time. The data set used for sentiment analysis is from Kaggle [12]. The data set contains the cryptocurrency news data from October 12, 2021 to December 19, 2023. The data is presented in a structured format, including news title, body text, source, subject, etc. News content is captured through the network. This study uses data from the first 50 days of historical to predict the next day's Bitcoin price, and normalizes the data using Min Max Scaler.

2.2. Proposed approach

Specifically, first, Bi-LSTM and GRU hybrid models are used to model the time series (showed in Figure 1). Second, FbProphet model is used to process multi seasonal data. Thirdly, BERT is used to extract emotional features from social media content, and VADER is used for emotional analysis. Fourthly, multimodal fusion is carried out, and attention mechanism is introduced to dynamically adjust the prediction output of the model according to information such as market sentiment fluctuations. Finally, weighted fusion is carried out to obtain the final forecast price.

Figure 1: The pipeline of the model (picture credit: original)

2.2.1. Time series modelling (Bi-LSTM & GRU)

In this study, Bi-LSTM and GRU are used for time series modeling. The traditional RNN method will have the problem of gradient disappearing, while the LSTM method solves this problem by introducing the control gate mechanism. Bi-LSTM is an extension of LSTM. It processes time series information forward and backward at the same time to efficiently process data with long-term dependence, such as the price of Bitcoin in this study. The formula is shown in (1).

\( {h_{t}}=[{h_{t}}(f),{h_{t}}(b)] \) (1)

where \( {h_{t}}(f) \) is the implicit state of forward LSTM. \( {h_{t}}(b) \) is the hidden state of reverse LSTM. [ \( {h_{t}}(f) \) , \( {h_{t}}(b) \) ] It is the combination of two hidden states, providing a richer time series. GRU is a simplified version of LSTM. It removes separate memory elements and some gates. It controls the information flow through reset gates and update gates, so that information can be retained or forgotten according to requirements and needs. Update gate formula as shown in (2); Reset gate as shown in Formula (3).

\( {z_{t}}=σ({W_{z}}\cdot [{h_{t-1}},{x_{t}}]+{b_{z}}) \) (2)

\( {r_{t}}=σ({W_{r}}\cdot [{h_{t-1}},{x_{t}}]+{b_{r}}) \) (3)

where \( {z_{t}} \) is the output of the update gate and \( {r_{t}} \) is the output of reset gate.

2.2.2. FbProphet model

FbProphet is an open-source time series prediction tool developed by Facebook. It is based on the additive model and combines the trend part, seasonal part, holiday effect and other factors of the time series, so it is very suitable for processing data with strong seasonality and periodicity. FbProphet is robust and can automatically handle extreme fluctuations and outliers without being disturbed by them, maintaining the stability of time series. This is very suitable for the long-term volatility of the Bitcoin market. It can reduce the influence of outliers by modifying the noise term, thus making the prediction of the model more stable and reliable. The formula is shown in (4).

\( y(t)=g(t)+s(t)+h(t)+∈(t) \) (4)

y(t): The value of the target variable (such as the price of Bitcoin) at time point t. g(t): Trend function, which represents the long-term change trend of data, usually linear or logistic growth. s(t): Seasonal component, representing repeated periodic changes, usually capturing seasonal fluctuations of year, week or day. h(t): Holiday effect, considering the impact of important holidays (such as festivals, major events, etc.) on data. ϵt: The noise term indicates unpredictable error or accidental fluctuation.

2.2.3. VADER

VADER is designed suitably for analyzing contexts’ emotion in social media. It divided the text into three categories by lexical expression rules and calculated the composite emotion score, which were positive (>=0.05), neutral (0.05~-0.05) or negative (<=-0.05). VADER calculates the overall emotion based on the emotion score of each word, without the need of marking data to train the model. This method is good at analyzing abbreviations, slang and emoticons, but it is often difficult to identify complex semantics such as irony and contextual changes, which occasionally leads to inaccurate judgments.

2.2.4. BERT

BERT is a pre training model based on Transformer encoder, which improves accuracy through deep neural network. After pre training, the deep and metaphorical features of contexts can be captured by BERT with ease. In the fine-tuning phase, only the output layer parameters need to be adjusted to ensure rapid convergence and high classification accuracy. At the same time, BERT performs well in emotional analysis, and is good at identifying complex emotional patterns, satire and context dependent emotional changes, which makes up for the shortcomings of VADER.

2.2.5. Attention mechanism

Attention mechanism is a selective mechanism that automatically focuses on an important part of input data and simulates human attention. The core of the attention mechanism is to let the model dynamically allocate weights according to the importance of the input content, so as to focus on the most useful information for prediction. The main steps are query, key and value, weight calculation and weighted sum. Facing the complex cryptocurrency market, the attention mechanism can effectively avoid the problem of gradient disappearance and adjust the weight according to market changes to improve the modeling credibility in this study.

3. Result and discussions

3.1. Technical analysis

This study first uses the Bi-LSTM and GRU hybrid model to predict Bitcoin price data in real time. From January 1, 2012 to March 1, 2025, 80% of the Bitcoin price data will be used as the training set and 20% as the test set. As can be seen from Figure 2, the traditional Bi LSTM and GRU hybrid models can only roughly predict the trend of Bitcoin price changes. The predicted price of Bitcoin is lower than the actual price and the difference is large.

Figure 2: Prediction with Bi-LSTM and GRU (picture credit: original)

In order to reduce the impact of seasonality and other special times on price forecasting, the framework used in this study combines the FbProphet model. As can be seen from Figure 3, after combining the FbProphet method, the predicted price of Bitcoin is closer to the actual price, and the outliers are reduced, showing strong robustness.

Figure 3: Prediction with Bi-LSTM, GRU and FbProphet model (picture credit: original)

3.2. Sentiment analysis

Out of profound study in other factors impacting on the Bitcoin’s market prices objective, another attention is put on the impact of public sentiment on the price of Bitcoin in this study. The text used is from three major cryptocurrency news websites, and the three emotional classifications are shown in Table 1. Real time sentiment is easy to have a dual impact on the market price of special currency, and the data is completely dependent on technical analysis. Because traditional technology is difficult to analyse irony and slang, this study combines VADER and BERT methods to improve the prediction ability of this model. The prediction results of Bitcoin prices are shown in Figure 4. Compared with Figure 3, the prediction accuracy has been significantly improved, but there are still shortcomings.

Table 1: Examples of different sentiment classification

Figure 4: Prediction with mixed model and sentiment model (picture credit: original)

3.3. Mixed model

According to the above results, the framework used in this study further combines the attention mechanism to dynamically adjust the weighted proportion of fusion prediction, and the results are shown in Figure 5. To sum up, this study can significantly improve the accuracy of bitcoin price prediction by combining multiple methods.

Figure 5: Prediction with final mixed model (picture credit: original)

3.4. Mixed model

The Bitcoin price prediction model proposed in this study integrates machine learning techniques with several innovative technologies, with the goal of enhancing prediction accuracy and assisting investors in navigating volatile market fluctuations. The model combines a hybrid deep learning architecture of Bi-LSTM and GRU, the FbProphet model for multi-seasonal data processing, BERT and VADER for emotion analysis, and an attention mechanism. In comparison to traditional methods, the model demonstrates a substantial reduction in prediction error, offering improved stability and accuracy, especially in the noisy and unpredictable market conditions characteristic of the cryptocurrency market. This approach provides more reliable predictions in complex and uncertain environments, thereby boosting investors' ability to manage risks and making more informed decisions in the ever-changing cryptocurrency market.

4. Conclusions

Digital cryptocurrencies, particularly Bitcoin, hold significant potential for future applications and growth. Accurately predicting cryptocurrency price trends is essential for investors to make well-informed decisions. This study combines various research methods, including time series forecasting with Bi-LSTM and GRU models, stabilization of the time series using FbProphet, and sentiment analysis of Bitcoin-related news via VADER and BERT. An attention mechanism is employed to dynamically allocate weights, improving the model's responsiveness to changes in the external environment. The proposed approach demonstrates lower error rates in predicting the price of digital cryptocurrencies, enhancing forecasting accuracy. Looking forward, future work will expand the model to include additional digital currencies to further validate its effectiveness and accuracy. Moreover, the influence of more indirect factors (e.g. international relations, gold and oil prices, and global policies) on cryptocurrency values will be considered. Ultimately, the aim is to integrate the predictive model into financial software, providing a convenient and reliable tool for investors to make informed decisions.