Detecting tension in online communities with computational Twitter analysis

•We collect and analyze data from the Twitter streaming API.•We combine text mining and computational conversation analysis to detect tension.•We compare the results to machine learning and sentiment analysis.•We outperform machine learning and sentiment analysis approaches.•Sentiment analysis' difference in mean maximum daily polarity can also indicate tension.

The growing number of people using social media to communicate with others and document their personal opinion and action is creating a significant stream of data that provides the opportunity for social scientists to conduct online forms of research, providing an insight into online social formations. This paper investigates the possibility of forecasting spikes in social tension – defined by the UK police service as “any incident that would tend to show that the normal relationship between individuals or groups has seriously deteriorated” – through social media. A number of different computational methods were trialed to detect spikes in tension using a human coded sample of data collected from Twitter, relating to an accusation of racial abuse during a Premier League football match. Conversation analysis combined with syntactic and lexicon-based text mining rules; sentiment analysis; and machine learning methods was tested as a possible approach. Results indicate that a combination of conversation analysis methods and text mining outperforms a number of machine learning approaches and a sentiment analysis tool at classifying tension levels in individual tweets.