Cross-domain collaboration for improved IDS rule set selection

Managing an intrusion detection system (IDS) requires careful consideration of the IDS rule set used to match malicious traffic. Network operators face a tradeoff when selecting rules: a rule set that is too conservative (too few rules) could lead to network intrusion and attacks from unforeseen risks, while a rule set that is too broad (too many rules) runs the risk of increasing false alerts and diminishing network throughput. The ultimate goal is to deploy rules that are conservative but proactive, and optimizing and testing such a rule set can be time consuming and limited when considering only locally observed network traffic. We argue that automated techniques to compare feedback from multiple collaborating sources, such as collaborative filtering between networks, can improve local rule sets. Our system, ROCK (Rule set Optimization via Collaborative Knowledge), recommends network-specific, locally untested rules to network operators based on correlations between their feedback and previously submitted feedback from other operators. We evaluated ROCK in two experimental deployments to detect shellcode and in simulation to measure the effect of broad collaboration. Network operators benefitted even if they provided feedback ratings for as few as 5 rules and deployed only the top 5 rules that ROCK recommended for their network; shellcode detection rates increase by up to 150% over a local baseline with little to no impact on false alerts. Our simulation analysis suggests that ROCK's recommendation quality increases rapidly with the number of user networks and can leverage varied degrees of similarity across networks. Our results demonstrate how security through collaboration can benefit local networks and provide proactive security in an automated way.