Bulletin of the American Physical Society
APS March Meeting 2022
Volume 67, Number 3
Monday–Friday, March 14–18, 2022; Chicago
Session M38: Quantum Characterization, Verification, and ValidationFocus Recordings Available

Hide Abstracts 
Sponsoring Units: DQI Chair: Gabriel Samach, MIT Room: McCormick Place W195 
Wednesday, March 16, 2022 8:00AM  8:12AM 
M38.00001: ApplicationOriented Performance Benchmarks for Quantum Computing (Part I) Jason Necaise, Paul R Varosy, Jeremiah D Coleman, Tom Lubinski, Sonika Johri, Luning Zhao, Charles H Baldwin, Karl Mayer, Timothy J Proctor In this work we introduce an open source suite of quantum applicationoriented performance benchmarks that is designed to measure the effectiveness of quantum computing hardware at executing quantum applications. These benchmarks probe a quantum computer's performance on various algorithms and small applications as the problem size is varied, by mapping out the fidelity of the results as a function of circuit width and depth using the framework of volumetric benchmarking. In addition to estimating the fidelity of results generated by quantum execution, the suite is designed to benchmark certain aspects of the execution pipeline in order to provide endusers with a practical measure of both the quality of and the time to solution. Our methodology is constructed to anticipate advances in quantum computing hardware that are likely to emerge in the next five years. This benchmarking suite is designed to be readily accessible to a broad audience of users and provides benchmarks that correspond to many wellknown quantum computing algorithms. 
Wednesday, March 16, 2022 8:12AM  8:24AM 
M38.00002: ApplicationOriented Performance Benchmarks for Quantum Computing (Part II) Jeremiah D Coleman, Paul R Varosy, Jason Necaise, Tom Lubinski, Sonika Johri, Luning Zhao, Charles H Baldwin, Karl Mayer, Timothy J Proctor In this work we introduce an open source suite of quantum applicationoriented performance benchmarks that is designed to measure the effectiveness of quantum computing hardware at executing quantum applications. These benchmarks probe a quantum computer's performance on various algorithms and small applications as the problem size is varied, by mapping out the fidelity of the results as a function of circuit width and depth using the framework of volumetric benchmarking. In addition to estimating the fidelity of results generated by quantum execution, the suite is designed to benchmark certain aspects of the execution pipeline in order to provide endusers with a practical measure of both the quality of and the time to solution. Our methodology is constructed to anticipate advances in quantum computing hardware that are likely to emerge in the next five years. This benchmarking suite is designed to be readily accessible to a broad audience of users and provides benchmarks that correspond to many wellknown quantum computing algorithms. 
Wednesday, March 16, 2022 8:24AM  8:36AM 
M38.00003: ApplicationOriented Performance Benchmarks for Quantum Computing (Part III) Paul R Varosy, Jeremiah D Coleman, Jason Necaise, Tom Lubinski, Sonika Johri, Luning Zhao, Charles H Baldwin, Karl Mayer, Timothy J Proctor In this work we introduce an open source suite of quantum applicationoriented performance benchmarks that is designed to measure the effectiveness of quantum computing hardware at executing quantum applications. These benchmarks probe a quantum computer's performance on various algorithms and small applications as the problem size is varied, by mapping out the fidelity of the results as a function of circuit width and depth using the framework of volumetric benchmarking. In addition to estimating the fidelity of results generated by quantum execution, the suite is designed to benchmark certain aspects of the execution pipeline in order to provide endusers with a practical measure of both the quality of and the time to solution. Our methodology is constructed to anticipate advances in quantum computing hardware that are likely to emerge in the next five years. This benchmarking suite is designed to be readily accessible to a broad audience of users and provides benchmarks that correspond to many wellknown quantum computing algorithms. 
Wednesday, March 16, 2022 8:36AM  9:12AM 
M38.00004: ApplicationMotivated, Holistic Benchmarking of a Full Quantum Computing Stack Invited Speaker: Daniel Mills Quantum computing systems need to be benchmarked in terms of practical tasks they would be expected to do. We propose 3 "applicationmotivated" circuit classes for benchmarking: deep (relevant for state preparation in the variational quantum eigensolver algorithm), shallow (inspired by IQPtype circuits that might be useful for nearterm quantum machine learning), and square (inspired by the quantum volume benchmark). We quantify the performance of a quantum computing system in running circuits from these classes using several figures of merit, all of which require exponential classical computing resources and a polynomial number of classical samples (bitstrings) from the system. We present results of these benchmarks, obtained using systems made available by IBM Quantum. We use these results to study how performance depends on choices made at several levels of the quantum computing stack, including the compilation strategy to use and the device on which the circuits are run. We show that noiseaware compilation strategies may be beneficial, and that device connectivity and noise levels play a crucial role in the performance of the system according to our benchmarks. 
Wednesday, March 16, 2022 9:12AM  9:24AM 
M38.00005: Reexamining the quantum volume test: Ideal distributions, compiler optimizations, confidence intervals, and scalable resource estimations Charles H Baldwin, Karl Mayer, Natalie C Brown, Ciaran RyanAnderson, David Hayes The quantum volume test is a fullsystem benchmark for quantum computers that is sensitive to qubit number, fidelity, connectivity, and other quantities believed to be important in building useful devices. The test was designed to produce a singlenumber measure of a quantum computer's general capability, but a complete understanding of its limitations and operational meaning is still missing. We explore the quantum volume test to better understand its design aspects, sensitivity to errors, passing criteria, and what passing implies about a quantum computer. We elucidate some transient behaviors the test exhibits for small qubit number including the ideal measurement output distributions and the efficacy of common compiler optimizations. We then present an efficient algorithm for estimating the expected success under different error models and compiler optimization options, which predicts performance goals for future systems. Additionally, we propose a new confidence interval construction that requires less measurements and demonstrate the savings with a QV=2^{10} experimental dataset collected from Honeywell System Model H1. Finally, we discuss what the test implies about a quantum computer's practical or operational abilities especially in terms of quantum error correction. 
Wednesday, March 16, 2022 9:24AM  9:36AM 
M38.00006: Quantum error mitigation with classical shadows ZePei Cian, Alireza Seif, Senrui Chen, Sisi Zhou, Liang Jiang Mitigating errors in quantum information processing devices is especially important in the absence of faulttolerance. An effective method in suppressing state preparation error is using multiple copies to distill the ideal component from a noisy quantum state. Here, we use classical shadows and randomized measurement to circumvent the need for coherent access to multiple copies at an exponential cost. We extensively study the scaling of resources using numerical simulations and find that the overhead is still favorable compared to full state tomography. We also apply our method to an experiment with trapped ions and successfully improve the fidelity of preparing a GHZ state. The analysis of the improved data also reveals the nature of errors affecting the experiment. Hence, our results provide a directly applicable method for mitigating errors in nearterm quantum computers. 
Wednesday, March 16, 2022 9:36AM  9:48AM 
M38.00007: OutofModel Effects and Overdispersion in Gate Set Tomography Corey I Ostrove, Erik Nielsen, Kevin C Young, Robin J BlumeKohout In real QCVV experiments, for example Gate Set Tomography (GST), it is normal to find that, even when fitting arbitrary models, there is significant evidence for outofmodel effects. In GST we use statistical testing to report our confidence in the presence of these effects. We propose a method for quantifying outofmodel effects which qualifies the predictions of the model by, instead of giving an output probability distribution for a quantum circuit, giving a distribution over probability distributions for that circuit. We refer to this as the DirichletMultinomial ansatz. In its simplest form this ansatz introduces a single additional parameter in our model called the "overdispersion parameter," which characterizes the spread of this probability distribution centered upon the predicted value of the output probabilities coming from GST. We've applied this to the reanalysis of a set of single qubit GST data and found that, even with a single additional parameter, this substantially reduces the model violation. Sandia National Labs is a multimission laboratory managed and operated by NTESS, LLC, a wholly owned subsidiary of Honeywell International Inc., for DOE's NNSA under contract DENA0003525. 
Wednesday, March 16, 2022 9:48AM  10:00AM 
M38.00008: Physical Model Gate Set Tomography Kevin C Young, Erik Nielsen, Kenneth M Rudinger, Brandon P Ruzic Gate set tomography (GST) has proven to be enormously successful for building predictive models of quantum information processor dynamics. But the process matrix models that are estimated by GST generally are described by a large number of free parameters that can be difficult to interpret. Connecting these process matrices to experimentally accessible parameters (such as laser intensity errors or magnetic field strength fluctuations) is an important step in improving devices, but is often done only in an ad hoc manner. In this talk, I'll discuss an extension of the GST framework that enables direct fitting of models for quantum devices that are expressed directly in terms of physically relevant quantities. These models often require expensive forward simulation, and so can be slow to compute and difficult to incorporate with iterative optimization routines. We overcome this with a caching and interpolation approach based on error generators. Our method enables resourceefficient GST experiments that can directly and accurately estimate experimental parameters. 
Wednesday, March 16, 2022 10:00AM  10:12AM 
M38.00009: Firstorder gaugeinvariant error rates in quantum processors Erik Nielsen, Kevin C Young, Robin J BlumeKohout Process matrix models for quantum gates operations contain nonphysical “gauge” degrees of freedom. These gauge freedoms wreak a surprising amount of havoc. For example, they imply that commonly used error metrics, such as the process fidelity and diamond distance of a quantum gate, are not physically welldefined quantities but rather depend on an arbitrary choice of reference frame. In this talk we present a partial solution to this problem by introducing error rates that are invariant under small gauge transformations to first order in the error strengths. These rates are useful in the common context where a quantum processor’s operations are close to ideal. We walk through a simple example, showing how firstorder gaugeinvariant rates can be associated with physically meaningful characteristics of a gate set and how they can be categorized into “intrinsic” errors associated with individual gates and “relational” errors that exist between sets of gates. 
Wednesday, March 16, 2022 10:12AM  10:24AM 
M38.00010: Benchmarking quantum coprocessors in an applicationcentric, hardwareagnostic and scalable way Thomas Ayral, Simon Martiel, Cyril Allouche Existing protocols for benchmarking current quantum coprocessors fail to meet the usual standards for assessing the performance of highperformancecomputing platforms. After a synthetic review of these protocols—whether at the gate, circuit, or application level—we introduce a new benchmark, dubbed Atos Qscore, which is applicationcentric, hardwareagnostic, and scalable to quantum advantage processor sizes and beyond. The Qscore measures the maximum number of qubits that can be used effectively to solve the MaxCut combinatorial optimization problem with the quantum approximate optimization algorithm. We give a robust definition of the notion of effective performance by introducing an improved approximation ratio based on the scaling of random and optimal algorithms. We illustrate the behavior of Qscore using perfect and noisy simulations of quantum processors. 
Wednesday, March 16, 2022 10:24AM  10:36AM 
M38.00011: Efficient diagnostics for quantum error correction Aditya Jain, Stephen D Bartlett, Joseph V Emerson, Pavithran Iyer Faulttolerant quantum computing will require accurate estimates of the resource overhead, but standard metrics such as gate fidelity and diamond distance have been shown to be poor predictors of logical performance. We present a scalable experimental approach based on Pauli error reconstruction to predict the performance of concatenated codes. Numerical evidence demonstrates that our method significantly outperforms predictions based on standard error metrics for various error models, even with limited data. We illustrate how this method assists in the selection of error correction schemes. 
Follow Us 
Engage
Become an APS Member 
My APS
Renew Membership 
Information for 
About APSThe American Physical Society (APS) is a nonprofit membership organization working to advance the knowledge of physics. 
© 2022 American Physical Society
 All rights reserved  Terms of Use
 Contact Us
Headquarters
1 Physics Ellipse, College Park, MD 207403844
(301) 2093200
Editorial Office
1 Research Road, Ridge, NY 119612701
(631) 5914000
Office of Public Affairs
529 14th St NW, Suite 1050, Washington, D.C. 200452001
(202) 6628700