Two-stage DOA estimation of independent and coherent signals in spatially coloured noise

• More fourth order cumulant matrices are employed to achieve a more stable orthogonality between the signal and noise subspaces.
• A source enumeration method is presented for identification of the independent signals from the coherent ones.
• The independent components are eliminated without loss of the coherent signal information.
• A new rank restoration technique, which makes use of all entries of the cumulant matrix of the coherent signals, is developed.

A novel fourth order cumulant-based matrix reconstruction algorithm is presented for direction of arrival (DOA) estimation when both independent and coherent signals coexist in the presence of spatially coloured noise. The method operates in two stages. By exploiting the orthogonality between subspaces, the DOA estimates of uncorrelated sources are first obtained, then the independent components are eliminated by reconstructing the corresponding cumulant matrix. Secondly, a series of matrix reconstructions are performed, then the remaining coherent sources are resolved from the reconstructed matrix which has been rank restored. The resultant algorithm can resolve more signals than array elements with higher accuracy and better classification of signal type than previous methods. Simulation results demonstrate the validity and efficiency of the proposed method.