Tomlinson-Harashima precoding minimizing total MSE with transmitter spatial correlation

This paper proposes a Tomlinson-Harashima precoding (THP) transceiver for multiple-input multiple-output (MIMO) system, where the spatial correlation information at the transmitter is included in the channel state information (CSI) model. It derives the total mean square error (MSE) and its lower bound as a function of precoding matrix. Then, a precoding matrix and the closed-form expression of minimum MSE lower bound are obtained by use of optimization and matrix theory. By right-multiplying a proper unitary matrix to the above precoding matrix, the paper develops the optimal precoding matrix, thus the optimal transceiver matrices are achieved. Simulation results show that the total MSE performance of the proposed method outperforms the existing linear method and the naive THP method.