Large scale opinion mining for social, news and blog data

Companies that collect and analyze data from social media, news and other data streams are faced with several challenges that concern storage and processing of huge amounts of data. When they want to serve the processed information to their customers and moreover, when they want to cover different information needs for each customer, they need solutions that process data in near real time in order to gain insights on the data in motion. The volume and volatility of opinionated data that is published in social media, in combination with the variety of data sources has created a demanding ecosystem for stream processing. Although, there are several solutions that can handle information of static nature and small volume quite efficiently, they usually do not scale up properly because of their high complexity. Moreover, such solutions have been designed to run once or to run in a fixed dataset and they are not sufficient for processing huge volumes of streamed data. To address this problem, a platform for real-time opinion mining is proposed. Based on prior research and real application services that have been developed, a new platform called “PaloPro” is presented to cover the needs for brand monitoring.