LSTM based stock prediction using weighted and categorized financial news (2024)

A significant correlation between financial news with stock market trends has been explored extensively. However, very little research has been conducted for stock prediction models that utilize news categories, weighted according to their relevance with the target stock. In this paper, we show that prediction accuracy can be enhanced by incorporating weighted news categories simultaneously into the prediction model. We suggest utilizing news categories associated with the structural hierarchy of the stock market: that is, news categories for the market, sector, and stock-related news. In this context, Long Short-Term Memory (LSTM) based Weighted and Categorized News Stock prediction model (WCN-LSTM) is proposed. The model incorporates news categories with their learned weights simultaneously. To enhance the effectiveness, sophisticated features are integrated into WCN-LSTM. These include, hybrid input, lexicon-based sentiment analysis, and deep learning to impose sequential learning. Experiments have been performed for the case of the Pakistan Stock Exchange (PSX) using different sentiment dictionaries and time steps. Accuracy and F1-score are used to evaluate the prediction model. We have analyzed the WCN-LSTM results thoroughly and identified that WCN-LSTM performs better than the baseline model. Moreover, the sentiment lexicon HIV4 along with time steps 3 and 7, optimized the prediction accuracy. We have conducted statistical analysis to quantitatively assess our findings. A qualitative comparison of WCN-LSTM with existing prediction models is also presented to highlight its superiority and novelty over its counterparts.

1.1 Research objectives

In order to improve the performance of news sensitive stock trend prediction, we have identified the following objectives of our research:

1. To propose and evaluate an extensive model, which can cater complexities of the hybrid input dataset in order to extract significant information.
2. To incorporate the filtered news groups into the prediction model simultaneously with their learned weight.
3. To effectively identify sentiments of news to predict the stock trend.
4. To adopt sequence learning in stock prediction to utilize historical sequence information.

The above research objectives lead towards the proposal of a Long Short-Term Memory (LSTM) based Weighted and Categorized News (WCN-LSTM) stock prediction model. WCN-LSTM integrates state of the art features. The proposed model utilizes hybrid input. It allows the incorporation of multiple weighted news groups simultaneously into the prediction model. WCN-LSTM performs feature extraction from news using lexicon-based sentiment analysis. It uses the LSTM layer to process sequential input data. The implementation of WCN-LSTM causes to raise the following empirical research questions.

1.2 Research questions

RQ1: Does news categorization give more insight into understanding news impact on the stock market in turn improving prediction accuracy?

RQ2: Do different news categories have different weights of their impact on the stock market to significantly enhance prediction performance?

RQ3: How to identify the optimized value of weights for categorized news impact?

RQ4: Which sentiment dictionary improves WCN-LSTM performance significantly? general or domain specific?

RQ5: What should be the optimized time step value of the input sequence for WCN-LSTM to predict the trend of the next day for a stock?

1.3 Contributions

We have proposed a new stock prediction model. Moreover, to demonstrate the significance of the proposed model in improving stock prediction accuracy our contributions are given below:

1. We have selected Pakistan Stock Exchange (PSX) in order to perform experiments. Stock prices are downloaded from the website of PSX Stock for the period of January 2006 to August 2018. News headlines are scraped from a newspaper (The News) archive along with the publishing date. News headlines are categorized using an unsupervised classification technique suggested by Usmani and Shamsi [11].
2. We have implemented our proposed stock prediction model, WCN-LSTM. Moreover, to perform a quantitative analysis, we have also implemented a baseline model proposed by Li, et al. [8].
3. We have performed extensive experiments by utilizing different sentiment lexicons and varying input sequence lengths, to reveal significant findings for the case of PSX.
4. We have adopted Wilcoxon signed-rank test to perform statistical analysis for all experimental scenarios (see section 6.2).
5. We have also presented a qualitative analysis between WCN-LSTM and existing stock prediction models to show the significance and novelty of our proposed model (see Fig 13).

For evaluation, we have adopted accuracy and F1-score as metrics. We have observed that our proposed approach performs better than the baseline approach by combining the impact of news categories according to their learned weights. The empirical research questions are answered by employing empirical and statistical evidence in the last section of the paper. The paper is organized as follows. Section 2 discusses the literature review. The proposed approach is discussed in section 3. The dataset description along with input preparation is presented in section 4. The experimental setup and results are discussed in sections 5 and 6, respectively. Finally, section 7 concludes this research along with future work. The organization of the paper is shown in Fig 1.

2.1 Stock prediction using categorized financial news

Mostly stock prediction deals with only one news category which influences the stock market. In literature, only few authors addressed the significance of incorporating categorized news in the prediction model. In [13], authors employed SVM for prediction and incorporated general market and company specific news along with technical indicators. They showed that prediction performance and system profitability were enhanced by taking in multiple news categories and technical indicators. However, they considered the news impact on the market for only 24 hours of its release. Although there could be some news that has long-term impact like news about government policies for the financial market.

There are different industry classification standards to group companies with a similar output to take the advantage of investigating their group effect. Global Industry Classification Standard (GICS) is one of the industry classification standards and employs four level hierarchy of Sector, Industry Group, Industry and Sub-Sector [2, 3].

Schumaker and Chen [2] employed GICS to partition news articles according to their relevance to sector, industry, subindustry, etc. They have used these news groups to identify their effect on stock prediction performance. They have found that prediction performance varies for different news group. However, they have used only one news group at a time to investigate its effect on prediction performance. For instance, the whole news dataset, sector-related news group, industry related news group etc. are incorporated in prediction model and they found that sector-related news group performed better among all news groups. Moreover, they employed Support Vector Regression (SVR) machine learning algorithm for prediction. Furthermore, news groups have not been incorporated simultaneously into the prediction model that is, the combined impact of all news groups has not been investigated in their work.

In [3], authors adapted this approach by incorporating all identified news groups simultaneously in the prediction model. They showed how properly weighted news articles with different degrees of relevance with stock prices used simultaneously can improve prediction performance significantly. They used the Multiple kernel learning (MKL) approaches for stock prediction by integrating the information coming from the prediction using a separate kernel for each news group. They have used newsgroups based on GICS standard which has some lacking in finding group of relevant articles.

In [18], the authors claimed that heterogeneity exists in GISC that limits the relevant finding regarding stock prediction. They proposed a model that searches for a group of companies with high relevance. They showed that the proposed model outperforms GISC system based prediction. However, these newsgroups are not belonging to the structural hierarchy of the stock market and their degree of relevance has not been learned ae well.

2.2 Stock prediction using sentiment analysis

Stock market-related news not only states the current market status but also has an impact on market volatility. There is a lot of work exists where market-related textual data is mined to extract sentiments about the financial market. There are two major ways to perform sentiment analysis: classification algorithm based and sentiment lexicon-based.

In the classification algorithm based approach labeled data is used to train the algorithm. Jiawei and Murata [6] used training data with positive and negative sentiment labels. LSTM is adopted for sentiment analysis and trend prediction. Multiple news articles at day t form an input sequence and are passed into the sentiment analysis module to generate a sentiment label at day t. Technical indicators after dimensionality reduction are passed along with sentiment labels into the trend prediction module. This module predicts the average trend of the next three days from day t and achieves 66.32% accuracy. Although they have proved the effectiveness of sentiment analysis by improving prediction performance, they have not utilized the strength of the LSTM model by passing input data of succeeding days. Carosia, et al. [7] performed tweets sentiment analysis using Multi-Layer Perceptron (MLP) for Brazilian stock market. Tweet sentiments are investigated using an absolute number of tweets, weighted tweets sentiments by favorites, and weighted tweets sentiments by retweets. Comparative analysis showed that MLP outperformed other machine learning models. They have considered only tweets sentiments as input data for market movement prediction which raises questions on prediction accuracy. Because there may be tweets about past moves and users may use multiple accounts for the same tweets.

Sentiment analysis can also be approached by using sentiment lexicons. These lexicons are created manually using rules and vocabulary [19, 20]. These manually created lexicons are small in size in turn limited the performance of prediction algorithms. In order to increase the size of sentiment lexicons, semi-automated approaches are proposed in the literature where manually created small-sized lexicons are used as seeds for automated approaches [21]. For instance, SentiWordNet 3.0 [22] and SenticNet 5 [23] are semi-automatically created sentiment lexicons. Sentiment lexicons are general and for a specific domain. For instance, Vader [19] and Harvard IV (HIV4) are general sentiment lexicons while LM is specific for the financial domain.

Li, et al. [24] adopted HIV4 and LM sentiment lexicons to generate news sentiment scores. They also used stock price data along with sentiment scores. Support Vector Machine (SVM) is used for making stock trend predictions. They have shown that sentiment scores enhanced prediction accuracy as compared to the Bag of Words (BoW) feature representation. However, the difference between the prediction accuracy of HIV4 and LM was insignificant. Although, it is mentioned that there are two groups of market-related and stock-related news. But they have not captured their impact separately so that deep insight could be explored.

Picasso, et al. [25] used technical indicators and news sentiment as input. They adopted a feed-forward neural network for stock trend prediction. News sentiment analysis is performed using LM and AffectiveSpace 2. They have examined the effectiveness of different feature sets. However, they have used a small-sized dataset not enough for comparative analysis.

Li, et al. [8] used four different manuals and semi-automatically created sentiment lexicons. They performed stock trend prediction for Hong Kong Stock Exchange. The proposed LSTM based stock prediction model incorporates stock prices, technical indicators, and news sentiment scores. They showed that hybrid input enhanced prediction accuracy. Moreover, sentiment lexicon specific for the financial domain performed better than other sentiment lexicon-based prediction performances. They have used LSTM layers in prediction model and incorporated sequential input data. But experiment to investigate the optimized time step for the input sequence was missing.

2.3 Stock prediction as a sequence learning problem

The problem of stock prediction is always challenging for the research community due to its high volatility. However, recent development in deep learning models opens new ways to tackle this type of data from different perspectives. For instance, Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), LSTM, etc. are efficiently used in literature.

Ding, et al. [26] extracted event-based textual features from the news. They improved the quality of extracted events using embedding and knowledge bases. They employed CNN for prediction and showed that their proposed approach for event extraction enhanced prediction accuracy. However, they have not taken advantage of stock price data along with textual features for stock trend prediction.

Vargas, et al. [14] input hybrid information into a prediction model based on hybrid deep learning approaches. Textual features are extracted using word and sentence embedding and technical indicators are derived from stock price data. The prediction model is built by combining CNN and LSTM layers and incorporating the previous day’s information in order to predict the stock trend of the next day. They showed that prediction accuracy is improved when technical indicators are also added to the input set along with the news titles. But they have not combined observations from successive days to form an input sequence that can be efficiently processed by LSTM.

Hu, et al. [9] incorporated hybrid information and utilized a hybrid deep learning model for news-oriented stock trend prediction. They adopted Gated Recurrent Unit (GRU) for sequential modeling. GRU is a variant of RNN. The authors [10], utilized stock price with a news sentiment score. They showed that analyzing the lengthy input sequences can significantly improve the accuracy of the LSTM based model. Li, et al. [8] input hybrid sequential information into the LSTM based prediction model. Pokhrel, et al. [27] performed a comparative study between CNN, LSTM, and GRU to predict closing price for Nepal Stock Exchange. They used Root Mean Square Error (RMSE) as a performance evaluation metric while the input sequence comprises stock prices, macroeconomics data, technical indicators and sentiment scores of financial news. They found that LSTM performed better than CNN and GRU. Li and Pan [28] used ensemble of LSTM and GRU deep learning models to learn from sequential data. They showed that ensemble learning significantly improves prediction performance. However, the above review in the context of the sequence learning problem shows that the optimized length of sequential data is not investigated for the problem under consideration. Moreover, hybrid deep learning models should be adopted to effectively enhance prediction performance.

In this discussion, we have found that previous approaches have recognized the significance of hybrid information for stock prediction. News processing using sentiment analysis based approaches have gained widespread interest in recent years because it allows to make faster and accurate conclusions. In stock prediction, sentiment analysis is mostly adopted by employing sentiment lexicons. However, research is going on for enrichment in sentiment lexicons by utilizing semi-automated and state of the art approaches.

News plays a vital role in stock prediction. Despite that, financial news categorization at a more granular level according to the structural hierarchy in the stock market is not addressed in the literature. Although, categorized news opens new perspectives to investigate news impact more deeply.

By addressing the research gap and considering the state of the art techniques, we propose an approach to predict the trend of the next day for target stock by employing information from previous days. Our proposed approach utilizes hybrid information as model input, employs sentiment analysis as a text mining technique, and incorporates categorized news into the prediction model along with their learned weights. Consecutive input vectors are combined to form a sequence according to a given time step and passed into the model. Moreover, the proposed model adopts LSTM layers to perform sequential learning. The proposed approach is discussed in the next section.

3.1 Problem statement

Given a stock s with close price, volume, technical indicators, market news sentiment score, sector news sentiment score, and stock news sentiment score over a lookback window of n days over the day range [t-n, t-1]. The stock trend for stock s from day t-1 to day t is defined as in Eq 1:(1)

We formulate the problem as in Eq 2:(2)

Where

δ = Stock close price, volume, and technical indicators

θ1 = Market-related news sentiment scores

θ2 = Sector-related news sentiment scores

θ3 = Stock-related news sentiment scores

α, β, and γ are weights for θ1, θ2, and θ3.

While(3)

Where

Trend = set of prediction labels

LSTM and Dense are neural network layers, used to predict stock trends.

The impact of financial news is equally important as the impact of stock price data in stock trend prediction. In our scenario, we have categorized financial news into three news groups according to the stock market structural hierarchy. Hence, there is a constraint that the total weight of categorized financial news has to be considered as 1 and the weight of each news category is learned through the training dataset. Eq 3 represents that sum of the total weights is equal to 1.

3.2 Architecture of the WCN-LSTM prediction model

The architecture of the WCN-LSTM is comprised of multiple neural network layers. In WCN-LSTM, each sequential input is passed to the LSTM layer. LSTM is a type of neural network deals with sequential data where the next observation is dependent on previous observations in a sequence. It can support short-term as well as long-term dependencies in a sequence by using its gating mechanism [5, 29]. In our scenario, the LSTM cell is adopted according to the baseline approach proposed in [8] where the sigmoid activation function is replaced with the hard sigmoid activation function.

The dropout layer tackles the issue of overfitting in the deep neural network which suffers from this issue due to the small dataset for training. In our model, the dropout layer is added after each LSTM layer so it probabilistically excludes some input vector and recurrent connections to LSTM [8]. The concatenate layer is responsible to concatenate the input vectors coming from different tracks into an output vector.

The dense layer is a neural network layer that is connected deeply, which means each neuron in the dense layer receives input from all neurons of its previous layer. The dense layer is found to be the most commonly used layer in the models. In our WCN-LSTM, it is the last layer and contains a sigmoid activation function which is specifically used for binary classification problems.

We have combined all layers discussed above and presented our proposed prediction model WCN-LSTM. The architecture of our proposed model is demonstrated in Fig 2.

4.1 News dataset

There are two ways to perform text categorization, by adopting automatic classification algorithms or manually assigning categories to each news. Classification algorithms required enough amount of labelled data for training [30, 31].

If there is no labelled data, manual effort is required to perform news categorization which is a tiring job. There are approaches that reduce manual efforts by introducing the semi-automatic way of text classification. These approaches manually define some domain related keywords, extend the list of keywords using NLP techniques, and then utilize these keywords in clustering methods [32, 33].

We are considering news headlines as textual input data rather than complete news articles for text categorization. The authors Chen, et al. [34] suggested that news headlines contain less noise and more valuable information than news bodies. In [14, 35, 36], news titles are suggested to perform experiments.

Publically available news headlines are scrapped from 2006 to 2018 and then arranged into three different groups. In order to categorize financial news headlines according to their area of influence, we considered the structural hierarchy of the stock market where the stock market has multiple sectors and each sector has multiple stocks.

In [11], news headlines related to Pakistan Stock Exchange (PSX) are filtered and categorized according to the structural hierarchy of the stock market using a proposed semi-automatic approach. News headlines are divided into three news groups. The first group contains news related to the whole stock market. The second group contains news related to the specific sector and the third group holds news related to a specific stock. News headlines are aligned with their publishing date. For any specific date, there might be news related to all three news groups or maybe only for one or two news groups. In Table 1, all news groups and their descriptions are given. PSX has many sectors but for this research work, a limited number of leading sectors are considered. News headlines related to these sectors are labeled according to the approach proposed in [11] for unlabeled data.

PSX or KSE-related news headlines belong to the first news group where news represents whole stock market. While all selected sectors-related news belongs to the second news group. Whereas, all active stocks in selected sectors belong to the third news group. In Table 2, all news categories along with their number of filtered news headlines is illustrated.

4.2 Stock price dataset

Stock price data contains different attributes like open price, close price, volume, etc., and is aligned with the publishing date. We have selected close price and volume as input features. Furthermore, the dataset is processed to derive some new attributes. We have to perform binary classification to perform stock trend prediction. So the target variable trend is calculated according to the Eq 1.

5.1 Baseline model

We have adopted a baseline model from [8] for the binary prediction of stock trends. Financial news is used to calculate sentiment scores without further categorization. Stock’s close price, volume, and technical indicators are used along with news sentiment scores. The input sequence generated from stock price data is the same as formulated in Eqs 4 and 5. The news sentiment score vector is termed as θ and the sequence generated from news sentiment scores is represented as in Eq 9:(9)

In the baseline model, two sequences of stock price data and news sentiment scores are passed as input into the model. Input sequences are passed into the concatenation layer where both sequences are combined and observations in sequences are aligned according to publishing date. Our WCN-LSTM model is adapted from the baseline model and detail related to the neural network layers have been discussed in section 3. The architecture of the baseline model is demonstrated in Fig 3.

5.2 Prediction models’ setup

In order to evaluate the neural network based prediction model, we have selected binary cross-entropy as a loss function in our binary classification problem. The neural network model’s training is performed by an optimizer algorithm. we have selected Root mean square prop (RMSProp), suggested by [37] for recurrent neural network. Learning rate is a critical hyper parameter of an optimizer algorithm. It defines the step size of each iteration in optimization algorithm while approaching the minima of loss function. Initially learning rate is set to 0.001. Moreover, a learning rate deduction technique is employed to adjust it according to the change of loss. After five consecutive epochs, if no change in loss, the learning rate is reduced to one tenth of itself [8]. Batch size and the maximum number of epochs are set to 32 and 500 for both prediction models.

5.3 Prediction models’ validation and optimization

In order to validate and optimize prediction model performance, we have performed model training using cross validation and adopted the grid search technique in search of optimizing the model’s hyper-parameters. We have also incorporated an early stop mechanism to improve generalization and to reduce overfitting of the deep learning model.

To accomplish the model tuning process, we have divided the dataset into three parts. The first part contains 70% of the total data and is used for model learning. While the remaining data is divided into two equal parts used for model testing and implementing an early-stop mechanism.

5.3.1 Time series split cross validation. In stock prediction, data related to the stock market is treated as a time series. That is observations are collected at regular intervals of time. Mathematically, it is described as in Eqs 5,6,7,8, and 15 for numerical and textual data. For time series data, the shuffling of data is incorrect to validate the model’s performance. Consequently, time series split cross validation is suggested in the literature by Ratto, et al. [38] for stock prediction. In a time split cross validation scheme, training and validation sets are selected in each iteration so that the validation set is always ahead of the training set. Likewise, we have adopted time series cross validation and divided the training set into 3-folds.

6.1 Experimental results

Experiments are performed for three different time steps: 3, 7, and 10 previous days’ information, using three different sentiment dictionary scores. For the baseline model, input sequences are generated using the selected time step and concatenated using date, and then passed to the first LSTM layer in the baseline model.

For WCN-LSTM input sequences are generated using the same approach but they are not concatenated. All these sequences are aligned according to the stock transactions and news publishing dates. These sequences are input to the model using a unique path for each sequence.

Experimental results are shown in graphs. Where columns represent stocks from all leading sectors of PSX and rows represent the accuracy of prediction models for each sentiment dictionary.

6.2 Statistical analysis

We have adopted Wilcoxon signed-rank test to statistically compare whether the predictive performance of the two models is significantly different from each other. The Wilcoxon signed-rank test is a non-parametric and distribution free technique. It is considered safer than a parametric t-test due to the exemption in the assumption of normality and hom*ogeneity of variance [39]. We have followed the work of [40–42] to use the Wilcoxon signed-rank test in order to comparatively analyze the predictive performance of WCN-LSTM and LSTM forecasting models.

We have constructed three hypotheses to statistically analyze the experimental results. For hypothesis testing, the model accuracy measure which is a continuous value is taken as a response variable. While for all three hypotheses, models, time steps, and sentiment lexicons are taken as independent variables. Our independent variable consists of two related groups where the same participants are presented in both groups. In order to conclude that one model performs better than the other, we have defined null and alternative hypotheses accordingly.

We have chosen 0.05 as a significant level or threshold value. To accept or reject the null hypothesis, the p-value is examined. If the p-value is less than a significant level, then the null hypothesis is rejected at a confidence level of 95%. The null hypothesis is accepted if the p-value is greater than the significance level. In the results, a p-value less than significance level is shown with ‘*’.

6.3 Discussion

We have implemented WCN-LSTM and LSTM based prediction models in order to answer our empirical research questions. We have also compared WCN-LSTM, qualitatively with existing models to highlight its novelty. Our comparative findings from different perspectives are discussed below:

S1 File.

https://doi.org/10.1371/journal.pone.0282234.s001

(PDF)