Robustness of Helical Edge States Under Proximate Band Reconstruction*

We analyze the edge structure of two-dimensional topological insulators described by the Bernevig-Hughes-Zhang model with the help of a self-consistent electrostatic modeling of sample edges, combined with a time-dependent perturbation theory approach. We find that for sufficiently smooth confinement, strongly interacting additional edge states arise under fairly general conditions. While their ground state is spin unpolarized, a spin-exchange coupling between helical and reconstructed edge states can lead to a dynamical spin polarization of the latter. However, we show that spatially random spin-orbit coupling inhibits such dynamical spin polarization and protects the helical edge states against backscattering. Further, we argue that at low electron density, the reconstructed edge states are well described by a Luttinger liquid.