Unusual behavior of the surface states in the topological insulator-magnetic insulator heterostructure

Topological insulator is a new class of solids that possess non-trivial topology in electronic structures. Spin-polarized conducting states should develop at the interface between topological insulators and trivial insulators as dictated by the topological invariants associated with the time-reversal symmetry. As such, the conducting surface states are very robust to impurities but susceptible to magnetic impurities that destroy their spin-momentum helical structures. We study the behavior of the surface states when magnetic impurity layers are deposited on top of the topological insulator surface, (Sb2Te3-MnTe), using first-principles calculations. We find that the helical nature of the surface states persists even at the presence of magnetic impurity layers and that the energy gap at the Dirac point due to the magnetic layers exhibits unusual behavior as the density of magnetic impurities is changed. We derive a model Hamiltonian based on Anderson model to describe the interaction of the surface states and magnetic-impurity layers and explain the behavior of the surface states. We show that the coupling between the surface states, d-orbitals of the magnetic impurities, and the RKKY-type interactions in magnetic impurities determine the energy gap of the surface states as well as the magnetic ordering in the impurity layer.