Quantifying event correlations for proactive failure management in networked computing systems

Networked computing systems continue to grow in scale and in the complexity of their components and interactions. Component failures become norms instead of exceptions in these environments. Moreover, failure events exhibit strong correlations in the time and space domains. In this paper, we develop a spherical covariance model with an adjustable timescale parameter to quantify the temporal correlation and a stochastic model to characterize spatial correlation. The models are further extended to take into account the information of application allocation to discover more correlations among failure instances. We cluster failure events based on their correlations and predict their future occurrences. Experimental results on a production coalition system, the Wayne State Computational Grid, show the offline and online predictions made by our predicting system can forecast 72.7–85.3% of the failure occurrences and capture failure correlations in a cluster coalition environment.

Research highlights
► Temporal and spatial failure correlations are modeled and quantified to characterize failure dynamics.
► A prediction mechanism that explores failure correlations is proposed to forecast future failure occurrences.
► High prediction accuracy is achieved in offline and online predictions on a production networked computer system.