Session F71: Hardware and Machine Learning for Fast Control

Optimal quantum control for transmons with Reinforcement Learning

Noise in existing quantum processors significantly limits their performance, washing out their quantum advantage. Methods based on quantum control and error mitigation can alleviate the impact of noise using real-time calibration and post processing of quantum output. We describe a reinforcement learning algorithm to optimize gate control pulses with built in error-mitigation specifically targeting leakage errors in superconducting transmon qubits. While reinforcement learning is model-free, we investigate the benefits of providing some noise model information to the agent. We train the reinforcement learning agent directly on noisy hardware by targeting an X gate as a proof of concept, and describe the adaptation of our algorithm to other gates. We compare our algorithm to existing control strategies, including the widely used DRAG formula. Our algorithm is designed with feasibility as a priority, meaning it has a low overhead compared to other control pulse optimization techniques. Improvements to the speed and fidelity of gate operations open the possibility for more extensive applications in quantum simulation, quantum chemistry and other algorithms on near-term and future quantum devices.   

Realizing a deep reinforcement learning agent discovering real-time feedback control strategies for a quantum system

To realize the full potential of quantum technologies, finding good strategies to control quantum information processing devices in real time becomes increasingly important. Usually these strategies require a precise understanding of the device itself, which is generally not available. Model-free reinforcement learning circumvents this need by discovering control strategies from scratch without relying on an accurate description of the quantum system. Furthermore, important tasks like state preparation, gate teleportation and error correction need feedback at time scales much shorter than the coherence time, which for superconducting circuits is in the microsecond range. Developing and training a deep reinforcement learning agent able to operate in this real-time feedback regime has been an open challenge.

Here, we have implemented such an agent in the form of a latency-optimized deep neural network on an FPGA. We demonstrate its use to efficiently initialize a superconducting qubit into a target state. To train the agent, we use model-free reinforcement learning that is based solely on measurement data. We study the agent's performance for high-fidelity, low-fidelity and three-level readout, and compare with simple strategies based on thresholding. This demonstration motivates further research towards adoption of reinforcement learning for real-time feedback control of quantum devices and more generally any physical system requiring learnable low-latency feedback control.   

Data-Driven Qubit Characterization and Optimal Control using Deep Learning

Making quantum computing practically viable requires the optimization of high-fidelity gate control pulses. One approach is the generation of control pulses based on numerical optimization. However, purely model-based approaches for offline-optimization face the difficulty that the qubit dynamics depend on many aspects of the experimental setup and device, many of which are difficult to characterize. Alternatively, closed-loop optimization schemes like Gate Set Calibration (GSC) [1], a protocol using linearized Gate Set Tomography, can be used for optimizing gates directly on the experiment. However, such approaches do not support the use of gradient-based optimization algorithms, because the estimation of gradients by finite differences can require too many measurements.

I will present a hybrid data driven method to address this problem, in which a surrogate model is obtained by training a RNN to predict the observed measurement outcomes after executing random pulses. Then, control pulses are obtained by using an GSC [1] inspired loss function as a proxy for the infidelity. Thus, the dynamics of the sampled device are represented with minimal prior models and the surrogate model provides methods to efficiently calculate gradients. This approach is verified in simulation.

[1] https://doi.org/10.1103/PhysRevApplied.13.044071   

Combining machine-learning characterization and quantum optimal control to improve superconducting qubit operations

Quantum optimal control theory offers a powerful toolbox to design pulse shapes that can realize, in numerical simulations, desired quantum operations with extremely high fidelity. When implementing these pulses in practice, however, the benefit of using optimal pulses over simple analytical forms is often greatly reduced. A significant part of this discrepancy can be attributed to failures of the numerical model to precisely capture the complete quantum dynamics generated by the control electronics. Here, we address this issue directly by building a framework where we break down the problem of realizing high-fidelity quantum operations into two parts. First, we use physics-inspired machine learning to infer an accurate model of the dynamics from experimental data. We investigate a range of trainable models from black-box neural networks to physically informative Lindblad master equation solvers. Second, we use such a trained numerical model in combination with state-of-the-art quantum optimal control to find pulse shapes that realize quantum gates with maximal accuracy given our experimental constraints. Using numerical simulations, we show the feasibility of learning from realistically available data to accurately characterize qubit dynamics and to discover high-fidelity arbitrary single-qubit gates. We then demonstrate our framework in an experimental setting by optimizing the Clifford gate set of a superconducting transmon qubit.   

Recommender System Expedited Quantum Control Optimization

Quantum control optimization algorithms are routinely used to synthesize optimal quantum gates or to realize efficient quantum state transfers. The computational resource required for the optimization is an essential consideration in order to scale toward quantum control of larger registers. Here, we propose and demonstrate the use of a machine learning method, specifically the recommender system (RS), to deal with the challenge of enhancing computational efficiency. Given a sparse database of a set of products and their customer ratings, RS is used to efficiently predict unknown ratings. In the quantum control problem, each iteration of a numerical optimization algorithm typically involves evaluating a large number of parameters, such as gradients or fidelities, which can be tabulated as a rating matrix. We establish that RS can rapidly and accurately predict elements of such a sparse rating matrix. Using this approach, we expedite a gradient ascent based quantum control optimization, namely GRAPE, and demonstrate the faster construction of two-qubit CNOT gate in registers with up to 8 qubits. We also describe and implement the enhancement of the computational speed of a hybrid algorithm involving simulated annealing as well as gradient ascent.

Moreover, the faster construction of three-qubit Toffoli gates further confirmed the applicability of RS in larger registers.   

Neural network accelerator for universal quantum control

Efficient quantum control is necessary for practical quantum computing implementations with current technologies. Conventional algorithms for determining optimal control parameters are computationally expensive, largely excluding them from use outside of the simulation. Existing hardware solutions structured as lookup tables are imprecise and costly. By designing a machine learning model to approximate the results of traditional tools, a more efficient method can be produced. Such a model can then be synthesized into a hardware accelerator for use in quantum systems. We demonstrate a machine learning algorithm for predicting optimal pulse parameters. This algorithm is lightweight enough to fit on a low-resource FPGA and perform inference with a latency of 175 ns and pipeline interval of 5 ns with > 0.99 gate fidelity [1]. We extend the workflow in Ref. [1] to a universal gate set and use the accelerator near quantum computing hardware where traditional computers cannot operate, enabling universal quantum control at a reasonable cost at low latencies without incurring large data bandwidths outside of the cryogenic environment.

[1] D. Xu et al., arXiv: 2208.02645, to appear in IEEE Conference Proceedings, International Workshop on Quantum Computing Software (SC22)   

Control, gates, and error correction of GKP codes in bosonic cQED

The past four years have seen rapid experimental progress in realizing the bosonic quantum error correction code proposed in 2001 by Gottesman, Kitaev, and Preskill (GKP), in which logical states are encoded as grid states of an oscillator [1]. Recent experimental milestones include the preparation and real-time error correction of the GKP code in trapped-ion and circuit quantum electrodynamics (cQED) architectures [2-6]. In this talk, I will review the bosonic cQED experiments that have led to some of these recent advances, focusing on the engineering of tools needed for error correction of the GKP code [2] and how the same tools can be used for universal control of an oscillator with weak dispersive coupling to a qubit [5]. I will also discuss our recent results on optimizing the error correction protocol using model-free reinforcement learning, leading to the demonstration of a fully error-corrected quantum memory with coherence beyond break even [6]. Finally, I will discuss our ongoing experimental work towards realizing single- and two-qubit gates on error-corrected GKP qubits.

[1] Gottesman, Kitaev, Preskill, PRA 2001; [2] Campagne-Ibarcq, Eickbusch, Touzard et al. Nature 2020 ; [3] Flühmann et al. Nature 2019 ; [4] de Neeve et al. Nature Physics 2022; [5] Eickbusch et al. Nature Physics 2022; [6] Sivak et al. in preparation 2022.

    

FPGA-based signal generation and readout of SNSPD signals for quantum communication

We perform the first experimental demonstration of using an FPGA-based radio frequency system-on-chip (RFSoC) architecture from Xilinx as the main electronics components of a quantum network by producing entangled photon-pairs and measuring its entanglement quality.  Using a standard entangled photon-pair source, we constructed a simple demonstrator experiment illustrating the use of the RFSoC-FPGA in photonic time-bin encoded quantum networks.

One key component in producing time-bin encoded photons is a radio-frequency pulse generator. We demonstrate that using the digital-to-analog converters of the FPGA is comparable to commercial arbitrary waveform generators (AWG) by characterizing the coincidence-to-accidental ratio (CAR) and the entanglement visibility of the entangled photon-pair source in the x-basis. We measured a CAR as high as 154, in agreement with the CAR measured with AWG and a high entanglement visibility of 95%.

Finally, we also demonstrate the use of the RFSoC-FPGA in the detection of the photon signals from superconducting nanowire single photon detectors (SNSPDs). We show that the CAR measured using the custom digitizer firmware and pulse-shape reconstruction software is comparable to that measured using commercial time-to-digital converter.   

Real-time Fast Feedback Experiment enabled by a Customized FPGA-Based Control System

A quantum control system is crucial for precise and reliable quantum computations. We developed a customized FPGA-based control system with the quantum control processor at its core. The control processor is implemented as a RISC-V Rocket Core extended with the Quantum Instructon Set Architecture (QUASAR) co-processor. QUASAR facilitates quantum program execution for high control operation speed and future scalability enabling real-time feedback experiments. We previously demonstrated our custom control system on a Xilinx FPGA that can be used for various quantum experiments with a superconducting quantum information processor. Here we show a mid-circuit measurement experiment that allows extending the qubit's lifetime through fast real-time feedback.   

QubiC update, the flexible full stack quantum bit control system

Superconducting qubits interact with the classical world through radio frequency pulse generation and detection. Quantum control systems generate these RF pulses to manipulate qubit states and also conduct RF probe and measurements to read out the qubits state. The QubiC is an open source quantum bit control system based on the software defined radio concept and utilizes advanced Field programmable gate array (FPGA) together with low noise analog front end modules. This system provides a full stack control solution from the hardware to the software API. This talk will update the recent developments of the QubiC system, including multiple FPGA platform support as well as new control features.

    

Distributed Processor for FPGA-based Superconducting Qubit Control Using QubiC

Implementing quantum circuits utilizing mid-circuit measurement and fast feedback requires flexible, low-latency classical control. To this end, we have developed a custom FPGA-based processor for QubiC, an open source platform for superconducting qubit control. The processor is distributed in nature, with one core per arbitrary waveform generator (AWG); simplifying the design and allowing for straightforward scaling. Each processor core implements an instruction set consisting of timed pulse commands as well as simple arithmetic and branching instructions. Our design also includes interfaces for synchronizing different cores and requesting/receiving (optionally processed) measurement results. Together, these features allow a user to program arbitrary control flow within a quantum circuit based on previous measurement results. In this presentation, we will detail our design, describe its integration with the QubiC control stack, and demonstrate the processor's capabilities with a simple mid-circuit measurement and feedback experiment using transmon qubits at the LBL AQT (advanced quantum testbed).   

Driving a two-qubit gate between transmons using digitally reconfigurable cryogenic CMOS control electronics.

Control electronics composed of CMOS circuits are of growing interest for next generation quantum computing systems. Here we present experimental results on a two-qubit cross resonance gate generated from a CMOS control chip, that is thermalized to the 4K stage of a dilution refrigerator. This low power digitally reconfigurable arbitrary waveform generator (DRAWG) is fabricated on 14 nm FinFET technology and has an observed power dissipation of 23 mW per channel while control is active. It uses a single side band direct conversion topology to generate output frequencies between 4.5 and 5.5 GHz, and a maximum power output of -18 dBm. The cryo-DRAWG was used to generate the single and two-qubit control pulses necessary for calibrating and characterizing a cross-resonance gate between transmon qubits. Measurement results include Hamiltonian tomography, qubit lifetime and dephasing, as well as single-qubit and two-qubit randomized benchmarking (RB). We demonstrate an error-per-Clifford rate of ~5E-4 for 1Q RB and ~2E-2 for 2Q RB.   

Control of a 2-qubit superconducting quantum processor unit-cell using a cryogenic CMOS integrated circuit

A challenge that must be overcome to enable a large-scale fault-tolerant quantum computer is the realization of the control system required to generate the millions of time-varying flux biases (Z) and microwave pulses (XY) for high-fidelity manipulation of the quantum processor. The control electronics used in O(100) qubit quantum computers are implemented with room-temperature board-level electronics that present obstacles to scaling with respect to their required size, power consumption, refrigerator wiring complexity, and per-qubit wiring transfer function variability. In contrast, the implementation of the control electronics with use of cryogenic electronics holds the promise of significantly ameliorating these obstacles. In this talk, we describe the design and characterization of a low-power prototype cryo-cmos 2-qubit XYZ controller for full-control of a Sycamore quantum processor unit-cell. The talk will describe the controller architecture; detailed measurement results, including benchmarking of single and two-qubit gate performance; and a discussion of the promise and challenges of implementing candidate cryo-electronics control systems.