Conjugate gradient algorithm for optimization under unitary matrix constraint

In this paper we introduce a Riemannian algorithm for minimizing (or maximizing) a real-valued function JJ of complex-valued matrix argument WW under the constraint that WW is an n×nn×n unitary matrix. This type of constrained optimization problem arises in many array and multi-channel signal processing applications.We propose a conjugate gradient (CG) algorithm on the Lie group of unitary matrices U(n)U(n). The algorithm fully exploits the group properties in order to reduce the computational cost. Two novel geodesic search methods exploiting the almost periodic nature of the cost function along geodesics on U(n)U(n) are introduced. We demonstrate the performance of the proposed CG algorithm in a blind signal separation application. Computer simulations show that the proposed algorithm outperforms other existing algorithms in terms of convergence speed and computational complexity.