News Sentiment Analysis Techniques

There are various methods to calculate news sentiment in finance. One common way is to search and count key words and phrases that contain positively or negatively charged language. These text strings can be placed into categories (positive, negative, neutral) and one can differentiate their strength or impact by assigning different weights. For example, the word “bankruptcy” can carry a stronger weight value than “lawsuit” even though they both might fall under the category “Negative”.

Although simplicity and intuition work in its favor, there are many challenges with this approach.

1. Define a relevant list of words and phrases. Journalists, analysts, and market participants use specific language when talking about stocks and expressing their views or recommendations. They use things like better than expected, missed earning target, downgraded from equalweight to underweight, etc. Many scholars and researchers in the field have used generic lists or libraries of positive and negative terms like Harvard’s General Inquirer, which often fail to capture key financial language, its use and meaning in finance.
2. Define a point-of-view (i.e. identify sentiment by company). Identifying key financial language is one challenge, associating it to a specific company is another. Firstly, you need an accurate named entity recognition approach in order to detect mentions or references to companies in text. For example, the company Bank of America Corp. is also known in the news by its Ticker (BAC), a CUSIP or ISIN like US0605051046, and common aliases like “BofA” or more recently “Bank of America Merrill Lynch”. Regularly, companies change their exchange listings, names, merge or get acquired, and therefore a sense of time and information of corporate actions is key in company detection. That way you find only references to companies by information that is accurate at the time of story publication. Also, dealing with false-positives and disambiguation is essential. For example, the French energy company Total S.A. can often be confused with the word “total”, which is very common in news stories.
3. Associate emotionally charged language with companies. Once you’ve identified a company and some key words or phrases, the challenge is to understand the context in which they are used. If a story mentions Bank of America, Lehman Brothers and the word “bankruptcy”, is this enough to say that the story is negative for both companies? Let’s say the headline read “Lehman Bros files for bankruptcy”. The body of the story contained a few paragraphs about the fourth-largest US investment bank then just filing for bankruptcy protection, dealing a huge blow to the fragile global financial system. In the very last paragraph, the story might say “In other news, BofA is up 1.5% for the day”. Without context and proper company identification, any assumptions for sentiment are flawed.
4. Map companies to traded securities. News stories normally make references to companies not securities. Therefore, one must have information about companies that is correct at each point in time, identifying them by ISINs and ticker symbols accurate at the time of story publication. References to companies in text must then be mapped to the proper traded securities identifiers.
5. Associate security-mapped, company-specific language with stock price movements. Perhaps the most challenging of all, it’s imperative to model and back test so-called positive and negative key words, phrases terms with subsequent positive and negative abnormal stock returns (or other directional benchmark metrics). I did some work on this area and documented the results here.
   

The work done by Paul Tetlock in More than Words: Quantifying Language to Measure Firms' Fundamentals addresses some of these challenges.

Other, perhaps more powerful news sentiment techniques include Expert Consensus and Market Response.

The Expert Consensus methodology entails training classification algorithms on the results of financial experts manually tagging thousands of stories and creating so-called “training sets”. Experts tag stories as likely to have a positive, negative, or neutral effect in the stock price of a given company in the hours ahead or on a given trading window. Stories where the majority of the experts agree on sentiment serve as consensus training sets. Classification systems can then be built using Bayesian networks or other probabilistic models to automate the process.

The Market Response methodology uses historical market data to measure the degree of impact a news item has on a specific market or security. A classifier can be trained on several years of news archives related to a set of global companies. The classifier is trained based on how markets tend to respond to news and assigns positive or negative ratings based on statistical analysis and not human judgment or input. Scholars from the University of Massachusetts have done some work related to this technique documented in the paper called Language Models for Financial News Recommendation.

In future postings, I plan to discuss these methodologies in more detail.

Construction of Market Sentiment Indices Using News Sentiment

This week I am going to present some ideas on how to construct sentiment indices for broader equity portfolios including for instance the Dow Jones Industrial Average (Dow 30), the Eurostoxx50 or some sector index. The resulting sentiment indices can be used as input to multifactor models or as part of the formulation of triggering events for high-frequency trading or longer-term investing.

Generally, market sentiment indices try to capture the prevailing sentiment trend for a particular index or sector. In order to capture such trend, it seems reasonable to consider an aggregation of news sentiment over a well-defined time window to capture the general "mood" of the market. When performing such aggregations it may be necessary to consider normalizing the data since equity news flow, as indicated in a previous posting, is characterized by strong seasonal patterns. One way of performing a sentiment aggregation is by considering net sentiment, which simply measures the count of positive minus the count of negative news items. Without normalizing for seasonality, it becomes difficult to evaluate what values reflect high and low net sentiment as for instance high values may simply be the result of certain bursts of information.

As an alternative to net sentiment, one may consider the sentiment ratio as the baseline for constructing market sentiment indices. As the ratio is measured as the count of positive to negative news items, it indirectly takes into consideration changes in news volume. This approach is expected to work well under the assumption that seasonality impacts the count of positive and negative news symmetrically.

In order to calculate the sentiment ratio for a particular index, one has to filter the data to obtain only news items that at least mentions the index constituents. In addition, further filtering based on the relevance or aboutness of the identified stories as they relate to the constituents of the index may be required to reduce noise in the signal. In a previous blog-posting, it was indicated that only about 20% of news stories hold high relevance, thus indicating that as much as 80% could be adding noise.

To arrive at the market sentiment index, it is necessary to perform a final mapping of the sentiment ratio into index values using a mapping schedule that reflects information about what level or cut-off point is considered neutral sentiment. Generally, large counts of positive and negative stories can be observed during both bull and bear markets, still with larger sentiment ratios in the former market environment, see Figure 1. Therefore, it may be somewhat challenging, independent of market data, to decide on the neutral sentiment position. Especially, it seems important to evaluate the sentiment ratio in a historical perspective in order to deal with this "sentiment bias".

Click To Enlarge

Figure 1: Sentiment ratio for the Dow Jones Industrial Average based on a six months backward-looking news aggregation window (without relevance filtering).

Assuming that the period January 2005 through April 2009 is a good representation of both bull and a bear markets, even though somewhat extreme, the neutral position could simply be measured as the mean ratio over this period. Similarly, the mapping into the positive and negative sentiment domain can be done based on the sentiment ratio volatility, which ensures that more extreme sentiment ratios maps into values closer to zero on the negative side and closer to 100 on the positive side, this in a symmetric fashion.

In Figure 2, I have included the price series of the Dow 30 index as well as the market sentiment index with and without relevance filtering. Relevance is calculated using a company score (0-100) where higher values indicate the identified firm plays a more prominent role in the news story. For example, Bank of America (BAC) would get a score of 100 in a story mainly about the firm reaching a legal settlement with another company, but a score of 5 or 10 if it was simply mentioned as a source in a story towards the end among many other companies.

Click To Enlarge

Figure 2: Sentiment index for the Dow Jones Industrial Average based on a six months backward-looking news aggregation window: (a) the Dow 30 index, (b) the sentiment index without relevance filtering, and (c) the sentiment index with relevance filtering.

In one of the coming postings, I will present some results on how well these sentiment indices capture the return of the relevant equity index over different investment horizons. Preliminary results indicate that using the relevance filtered index as compared to the non-filtered index improves the correlation between market return and the sentiment index by a factor of three.

Data Source: RavenPack