Performance analysis of hierarchical group key management integrated with adaptive intrusion detection in mobile ad hoc networks

We develop a mathematical model to quantitatively analyze a scalable region-based hierarchical group key management protocol integrated with intrusion detection to deal with both outsider and insider security attacks for group communication systems (GCSs) in mobile ad hoc networks (MANETs). Our proposed adaptive intrusion detection technique is based on majority voting by nodes in a geographical region to cope with collusion of compromised nodes, with each node preloaded with anomaly-based or misuse-based intrusion detection techniques to diagnose compromised nodes in the same region. When given a set of parameter values characterizing operational and environmental conditions, we identify the optimal intrusion detection rate and the optimal regional area size under which the mean time to security failure of the system is maximized and/or the total communication cost is minimized for GCSs in MANET environments. The tradeoff analysis in performance versus security is useful in identifying and dynamically applying optimal settings to maximize the system lifetime for scalable mobile group applications while satisfying application-specific performance requirements.