Topological phase transition in anisotropic square-octagon lattice with spin-orbit coupling and exchange field

We investigate the topological phase transitions in an anisotropic square-octagon lattice in the presence of spin-orbit coupling and exchange field. On the basis of the Chern number and spin Chern number, we find a number of topologically distinct phases with tuning the exchange field, including time-reversal-symmetry-broken quantum spin Hall phases, quantum anomalous Hall phases and a topologically trivial phase. Particularly, we observe a coexistent state of both the quantum spin Hall effect and quantum anomalous Hall effect. Besides, by adjusting the exchange filed, we find the phase transition from time-reversal-symmetry-broken quantum spin Hall phase to spin-imbalanced and spin-polarized quantum anomalous Hall phases, providing an opportunity for quantum spin manipulation. The bulk band gap closes when topological phase transitions occur between different topological phases. Furthermore, the energy and spin spectra of the edge states corresponding to different topological phases are consistent with the topological characterization based on the Chern and spin Chern numbers.