Analytical approach to the edge state of the Kane-Mele model*

We investigate the edge state of a two-dimensional topological insulator based on the Kane- Mele model. We consider the two semi-infinite honeycomb lattices with a zig-zag and an armchair boundary, respectively. We construct the effective Hamiltonians for the edge states assuming exponentially decaying wave functions. With the boundary conditions for the both types of the boundaries, we derive the self-consistent equations for the energies and the decaying factors of the edge states. The numerical solutions of the self-consistent equations exhibit intriguing spatial behaviors of the edge states with respect to the spin-orbit coupling and the sub-lattice potential. We found the bifurcation behavior of the edge state width with respect to the sub-lattice potential in zigzag boundary. The bifurcation behavior discriminates the boundary dependencies of the edge state properties. We also discuss the relation between the sample size and the interaction parameters in the phase transition from normal insulator to topological insulator.