An electrical polar discontinuity presents an opportunity to stabilize an emergent phase at an interface. Here, we grow thin films and superlattices of dielectric DyScO₃ and ferroelectric BiFeO₃, i.e. (BiFeO₃)_m/(DyScO₃)_n for various m on (110)-oriented DyScO₃ single crystal substrates using pulsed laser deposition. The ferroelectric polarization in contact with the dielectric layer leads to a large electric field build-up in the absence of a compensating charge. We report the results of Scanning Transmission Electron Microscopy (STEM), X-Ray Diffraction (XRD) and Atomic-Force Microscopy (AFM) measurements, demonstrating that our films exhibit excellent structural quality. Atomic-scale images of the structure using STEM measurements shows that, for layer thicknesses of 4 nm (10 unit cells), BiFeO<span style="font-size:10.8333px">3</span> adopts an orthorhombic, anti-ferroelectric ground state. This work thus provides the first direct observation of this ground state in un-doped BiFeO₃, demonstrating the ability of interfacial field to dramatically change the structural and electric ground state of an ultrathin film.