An efficient off-grid DOA estimation approach for nested array signal processing by using sparse Bayesian learning strategies

•A sparsity-inducing DOA estimation method applied in nested array is proposed.•The proposed method is based on a Sparse Bayesian learning perspective.•The off-grid effects inherent in sparse recovery framework are taken into account.•The unknown noise variance can be eliminated easily.•The off-grid distances can be calculated by using a polynomial rooting procedure.

In this paper, we address the problem of sparsity-inducing direction-of-arrival (DOA) estimation in the context of array covariance vectors when a recently proposed nested array structure is utilized. The proposed approach is based on a Sparse Bayesian learning (SBL) perspective while the off-grid effects inherent in traditional sparse recovery algorithms are taken into account simultaneously. Specifically, the notable advantages of the developed approach can be attributed to two aspects. First, through a linear transformation, the unknown noise variance can be eliminated effectively and the estimation error between the actual and sample covariance matrix can be normalized to an identity matrix, thereby facilitating a noise-free sparse representation. Second, a computationally tractable polynomial rooting procedure is introduced to calculate the discretized sampling grid error one by one based on the reconstructed spatial power spectrum, thus this novel algorithm can maintain high estimation accuracy even under a coarse sampling grid. Extensive simulation results finally demonstrate the superiority of the proposed approach in terms of DOA estimation precision and computational efficiency over off-the-shelves techniques.