Topological Phase Transitions in a Hybridized Three-Dimensional Topological Insulator*

As the three-dimensional (3D) topological insulator (TI) approaches its 2D thickness limit, quantum tunneling between top and bottom surfaces turns their gapless Dirac band into a gapped state at the Dirac points. Analytical formulation suggests that the hybridization gap scales exponentially with a decrease in number of layers while the system oscillates between topologically trivial and non-trivial insulators. This work explores the transport properties of a 3D TI in the inter-surface hybridization regime. By experimentally probing the hybridization gap as a function of TI thickness using three different methods, namely thermal activation, differential conductance, and quantum capacitance, we map the crossover from 3D TI to 2D insulating state. We detect gap-closing features in the moderate hybridization regime with a perpendicular electric field, suggesting topological phase transitions in the regime. In certain parameter spaces of the non-trivial insulator, we observe quantization of the longitudinal conductance at 2e²/h indicating the quantum spin Hall state.