Efficient DOA estimation of noncircular signals in the presence of multipath propagation

The noncircularity of uncorrelated signals can improve the performance of direction of arrival (DOA) estimation using antenna arrays. However, due to reflection from surfaces or malicious jamming, the received noncircular signals may be a mixture of uncorrelated and coherent components in practice, which has not been considered by existing algorithms. This paper addresses the issue of DOA estimation for noncircular signals under multipath fading in smart antenna systems, and a novel two-stage processing method is proposed accordingly. By expanding the signal subspace from augmented array output, the DOA estimates of uncorrelated sources are first obtained, and then the covariance matrix of uncorrelated signals is reconstructed and removed from that of the mixed signals afterwards. Secondly, a spatial smoothing technique is performed on a denoised covariance to restore rank deficiency in the covariance of the remaining coherent signals. The resultant algorithm can resolve more signals than array elements with higher accuracy and better classification of signal type than previous methods. Additionally, a new deterministic direction estimation Cramér-Rao lower bound (CRLB) is derived for the considered mixture model. Simulation results demonstrate the validity and efficiency of the proposed method.