Zigzag order and phase competition in expanded Kitaev–Heisenberg model on honeycomb lattice

• Expanded Kitaev–Heisenberg model on honeycomb lattice is investigated.
• Kitaev interactions between the first or second nearest neighbors are considered.
• Phase competition is discussed by energy calculation and Monte Carlo simulation.
• Zigzag phase shows a symmetric behavior to the stripy phase.
• Zigzag order is extended to the whole parameter range by more interactions.

The Kitaev–Heisenberg model on the honeycomb lattice is investigated in two cases: (I) with the Kitaev interaction between the nearest neighbors, and (II) with the Kitaev interaction between the next nearest neighbors. In the full parameter range, the ground states are searched by Monte Carlo simulation and identified by evaluating the correlation functions. The energies of different phases are calculated and compared with the simulated result to show the phase competition. It is observed from both energy calculation and the density of states that the zigzag order shows a symmetric behavior to the stripy phase in the pure Kitaev–Heisenberg model. By considering more interactions in both cases, the energy of zigzag order can be reduced lower than the energies of other states. Thus the zigzag phase may be stabilized in more parameter region and even extended to the whole parameter range.