On the convergence of the LMS algorithm with a rank-deficient input autocorrelation matrix

In all books and papers on adaptive filtering, the input autocorrelation matrix Rxx is always considered positive definite and hence the theoretical Wiener–Hopf normal equations (Rxxh=rxd) have a unique solution h=hopt (“there is only a single global optimum”, [B. Widrow, S. Stearns, Adaptive Signal Processing, Prentice-Hall, 1985, p. 21]) due to the invertibility of Rxx (i.e., it is full-rank). But what if Rxx is positive semi-definite and not full-rank? In this case the Wiener–Hopf normal equations are still consistent but with an infinite number of possible solutions.Now it is well known that the filter coefficients of the least mean square (LMS), stochastic gradient algorithm, converge (in the mean) to the unique Wiener–Hopf solution (hopt) when Rxx is full-rank. In this paper, we will show that even when Rxx is not full-rank it is still possible to predict the (convergence) behaviour of the LMS algorithm based upon knowledge of Rxx, rxd and the initial conditions of the filter coefficients.