Quantum fourier vision transformer circuits for variational self-attention-based learning*

The widely popular transformer network popularized by the generative pre-trained transformer (GPT) has a large field of applicability, including predicting text and images, classification, and even predicting solutions to the dynamics of physical systems. In the latter context, the continuous analog of the self-attention mechanism at the heart of transformer networks has been applied to learning the solutions of partial differential equations and reveals a convolution kernel nature that can be exploited by the Fourier transform. It is well known that quantum algorithms that have provably demonstrated a speedup over classical algorithms all have in common their utilization of the quantum Fourier transform. In this work, we explore quantum circuits that can efficiently express the self-attention mechanism through the perspective of the operator learning in the continuous analog of the self-attention mechanism. In this perspective, we are able to represent deep layers of a vision transformer network using simple gate operations and a set of multi-dimensional quantum Fourier transforms. We analyze the computational and parameter complexity of our novel variational quantum circuit and demonstrate its utility on a variety of datasets.