Automatic generation of HTTP intrusion signatures by selective identification of anomalies

In this paper, we introduce a novel methodology to automatically generate HTTP intrusion signatures for Network Intrusion Detection Systems (NIDS). Our approach relies on the use of a service-specific, semantic-aware anomaly detection scheme that combines stochastic learning with a model structure based on the protocol specification. Each incoming payload for the target service is tagged with an anomaly score obtained from probabilistically matching it against the corresponding learned model of normal usage. For those payloads whose anomaly score exceeds a given threshold, a more detailed analysis is performed to extract the portions that contribute the most to the anomaly score. Such portions are then used to build up candidate intrusion signatures, using a merging process that combines them with already existing patterns in order to keep the signature database as simple as possible by avoiding redundancies.We report results obtained with a specific implementation of our proposal for web traffic. During our evaluation, we used a well-known signature-based NIDS that sits behind the anomaly detection system and is fed with the signatures automatically generated by the latter. Our results indicate that functioning in such a way translates into an improvement of the often tedious signature generation process. Furthermore, a visual inspection of the signatures reveals that the generation procedure is quite reliable, mimicking (and, in some cases, even improving) attack patterns manually generated by security analysts. This results in an increase of the overall detection performance of the composite signature- plus anomaly-based system.