Partial distortion entropy maximization for online data clustering

Competitive learning neural networks are regarded as a powerful tool for online data clustering to represent a non-stationary probability distribution with a fixed number of weight vectors. One difficulty in practical applications of competitive learning neural networks to online data clustering is that most of them require heuristically-predetermined threshold parameters for balancing a trade-off between convergence accuracy, i.e. error minimization performance, and speed of adaptation to the changes in source statistics. Although adaptation acceleration is achievable by relocating a “useless” node so that it becomes useful, excessive relocation often disturbs error minimization. Hence, both of the adaptation speed and the error minimization performance sensitively depend on threshold parameters to determine whether a node should be relocated or not. In general, it is difficult to know adequate threshold parameters a priori. This paper proposes a novel criterion for decision making of node relocation without heuristically predetermined thresholds. According to the proposed criterion, a node is relocated only if the relocation task improves partial distortion entropy, which is an online optimality metric reliable from the viewpoint of error minimization. Hence, node relocation is carried out without disturbing error minimization. As a result, both quick adaptation and error minimization are simultaneously accomplished without any carefully predefined parameters. Experimental results clarify the validity of the proposed criterion. Competitive learning with the criterion is clearly superior to other representative algorithms in terms of both quick adaptation and error minimization performance.