Validating models of quantum computer performance*

Modeling low-level components of quantum computers is critical to understand quantum systems, identify errors, and pursue opportunities for engineering improvements. Tantamount to these tasks is the expectation that an effective and useful model should provide accurate predictions of circuit outcomes. Successfully predicting circuit outcomes validates our model and understanding of our quantum system, whereas failing to predict circuit outcomes indicates either a poor, inappropriate, or obsolete characterization. While evaluating the performance of a model is vital, it is not a binary metric and determining when a model is performing well or at least satisfactorily can be difficult.

We present a generalized process of model validation. We show how to determine and report performance on circuit prediction tasks using statistical tests and simulation. We demonstrate this process in experiment using gate set tomography and randomized benchmarking and evaluate performance on several distinct circuit prediction tasks. We find that our validation process is an effective tool in identifying model shortcomings and upgrading methodologies to create more accurate characterizations of quantum devices.