Topography and Mimicry of a Spin Liquid on a Triangular Lattice*

Motivated by recent interest in strongly-anisotropic spin-spin interactions, we have explored an extended three-dimensional phase diagram of a class of such models on an ideal triangular lattice using density-matrix renormalization group and quasiclassical approaches. In the case appropriate for the rare-earth-based materials with interactions only among the nearest-neighbor spins, we have mapped out all phases and identified the topography of the region that can harbor a spin liquid. A four-dimensional extension of this phase diagram with the next-nearest-neighbor coupling J₂ allows for a natural continuity of the spin liquid in the strongly-anisotropic model to the previously well-studied spin-liquid state of the J₁-J₂ Heisenberg model. Their respective spin-liquid states are shown to exhibit nearly identical spin-spin correlations, suggesting that the nature of spin liquids in both models is the same.
For YbMgGaO₄, an excellent material realization of the rare-earth-based triangular-lattice antiferromagnet with the highly anisotropic effective spin-spin interactions, our analysis finds no transitions to a spin liquid near experimentally relevant range of parameters, putting YbMgGaO₄ firmly in the stripe-ordered state. At the same time, in the regime with small or subleading anisotropic pseudo-dipolar terms, stripe states are selected by an order-by-disorder fluctuations, making stripes fragile toward orientational disorder. Then, the randomization of the pseudo-dipolar interactions due to spatially-fluctuating charge environment of the magnetic ions can successfully mimic a spin liquid by forming short-range stripe or stripe-superposition domains, producing the structure factor that is in agreement with experiment. This spin-liquid mimicry scenario is relevant to other quantum magnets with fragile ground states and random environments.