Iterative sIB algorithm

Recent years have witnessed a growing interest in the information bottleneck theory. Among the relevant algorithms in the extant literature, the sequential Information Bottleneck (sIB) algorithm is recognized for its balance between accuracy and complexity. However, like many other optimization techniques, it still suffers from the problem of getting easily trapped in local optima. To that end, our study proposed an iterative sIB algorithm (isIB) based on mutation for the clustering problem. From initial solution vectors of cluster labels generated by a seeding the sIB algorithm, our algorithm randomly selects a subset of elements and mutates the cluster labels according to the optimal mutation rate. The results are iteratively optimized further using genetic algorithms. Finally, the experimental results on the benchmark data sets validate the advantage of our iterative sIB algorithm over the sIB algorithm in terms of both accuracy and efficiency.

Research highlights
► We analyze the sequential Information Bottleneck algorithm (sIB) for the clustering problem. We identify its two major problems: the randomness in results and the insufficient optimization.
► To tackle the above problems, we propose an iterative sequential Information Bottleneck algorithm (isIB) based on genetic mutation for the clustering problem.
► We analyze the impact of the mutation rate on the performance of the isIB algorithm. We offer an optimal mutation rate via empirical studies.
► We perform a comprehensive experimental evaluation of isIB and sIB over a number of benchmark datasets with various criteria of accuracy and efficiency. The results indicate that isIB not only provides statistically better clustering solutions than sIB, but also needs less running time.