Local electronic structures and 2D topological phase transition of ultrathin Sb films*

We investigate local electronic structures of ultrathin Sb islands and their edges grown on Bi$_2$Te$_2$Se by scanning tunneling microscopy/spectroscopy (STM/STS) and density functional theory (DFT) calculations. The Sb islands of various thickness are grown with atomically well ordered edge structure over the 3 bilayers (BL). On the surfaces and edges of these islands, we clearly resolve edge-localized electronic states by STS measurements, which depend on the thickness. The DFT calculations identify that the strongly localized edge states of 4 and 5 BL films correspond to a quantum spin Hall (QSH) states while the edge states of 3 BL are trivial. Our experimental and theoretical results confirm the 2D topological phase transition of the ultrathin Sb films from trivial to QSH phase.