A relative trust-region algorithm for independent component analysis

In this paper we present a method of parameter optimization, relative trust-region learning, where the trust-region method and the relative optimization [M. Zibulevsky, Blind source separation with relative Newton method, in: Proceedings of the ICA, Nara, Japan, 2003, pp. 897–902] are jointly exploited. The relative trust-region method finds a direction and a step size with the help of a quadratic model of the objective function (as in the conventional trust-region methods) and updates parameters in a multiplicative fashion (as in the relative optimization). We apply this relative trust-region learning method to the problem of independent component analysis (ICA), which leads to the relative TR-ICA algorithm which turns out to possess the equivariant property (as in the relative gradient) and to achieve faster convergence than the relative gradient and even Newton-type algorithms. Empirical comparisons with several existing ICA algorithms demonstrate the useful behavior of the relative TR-ICA algorithm, such as the equivariant property and fast convergence.