Hysteretic magnetoresistance and unconventional anomalous Hall effect in the frustrated magnet TmB4

We study TmB$_4$, a frustrated magnet on the Archimedean Shastry-Sutherland lattice, through magnetization and transport experiments. The lack of anisotropy in resistivity shows that TmB$_4$ is an electronically three-dimensional system. The magnetoresistance (MR) is hysteretic at low-temperature even though a corresponding hysteresis in magnetization is absent. The Hall resistivity shows unconventional anomalous Hall effect (AHE) and is linear above saturation despite a large MR. We suggest that both hysteretic MR and AHE arise from the formation of complex non-coplanar structures at magnetic domain walls.