Tackling the Challenges of Situational Awareness Extraction in Twitter with an Adaptive Approach

Twitter is widely perceived as a potential source of valuable information for responders to mass emergencies. Despite interest in the development of extraction systems for such information, little effort has been put towards systemic methods for obtaining all posts pertaining to a disaster from the live Twitter stream. Researchers rely on keyword-based filters to extract information in spite of evidence that such markers are absent in many informational tweets, and also neglect the topic and traffic dynamics of the relevant tweets as crises progress. Previous work has shown that such practices can often lead to the loss of critical information in the context of a disaster. We introduce an adaptive filter, tailored to the idiosyncrasies of the real-time Twitter feed, intended to extract disaster-related content. Furthermore, we introduce a novel data model based on a three-label classification scheme to describe the composition of the data-stream. We use this model to simulate Twitter streams, modelling various post-disaster scenarios, for the purpose of filter performance evaluation. The filter is able to remove over 85% of the non-crisis content, and achieves a three-fold reduction in the loss of relevant contents compared to the existing approaches. In combination, the method and the model are useful tools for extracting situational awareness and highlight important directions for future work in this area.