Session BM4: Workshop II: State of the Art in Validation for Low Temperature Plasma Simulations and Experiments

Live

In the end, validation of a verified code is relied upon for claiming predictivity, a critical need for plasma simulation in many areas, especially involving safety or financial consequences. Here, verification means the code has been demonstrated to behave in a correct manner (i.e., convergence), sometimes phrased ``solving the equations correctly''. Validation means the code output has been shown to match experimental results, sometimes phrased ``solving the correct equations''. Precisely what ``predict'' means, and what makes for ``good'' or ``insufficient'' validation, are subjects of debate and evolving understanding in our community. Validation is rarely performed in a strong formal manner, although some examples exist. Instead, various kinds of evidence are used (sometimes indirectly) to formulate a story of confidence in results. Stronger validation leads to more confidence. This workshop will provide experiences and opinions of experts in validation and provide examples of validation exercises, including the sometimes unanticipated large uncertainties when formalism is applied to the process.   

Live

Validation experiments are experiments specifically designed to assess the predictive capability of a model. They differ from traditional experiments which are usually conducted to explore a poorly understood physical phenomena or to assess system performance. A key component of a validation experiment is the careful characterization of all required model inputs as well as their uncertainties. When these input uncertainties are propagated through a model, the resulting output is no longer a single number, but instead a nondeterministic outcome, usually a probability distribution. The subject of validation metrics addresses techniques for comparing these nondeterministic model outcomes to the experimentally measured values (and their uncertainties) and is currently an active area of research. This talk will discuss a number of approaches to validation, some of which focus on estimating model form uncertainty and others which focus more on model calibration. The question of how to infer model accuracy for conditions where no validation data are available will also be introduced.   

Live

Recent benchmarking activity shows that modern particle-in-cell simulations give results consistent with each other. This result suggests, but does not prove, that the results of these calculations are also correct, in some useful sense, and should be expected to be in agreement with relevant experiments. For the rare gases helium and argon, this expectation is strengthened by the good agreement that is found with experimental measurements of transport coefficients for both electron and ions. Of course, comparison with experiments involves complications such as the possible effect of charged particle emission from surfaces, multi-step ionization and geometrical factors. Some of these involve appreciably uncertain parameters, and the consequences of such uncertainty must be taken into account. In this paper, we will investigate whether there is, in light of these considerations, satisfactory agreement between particle-in-cell computer simulations and various now classical experimental studies. We will show that, in at least some cases, there apparently is not, and we will offer speculations as to why this might be the case.   

Live

A high degree of confidence in simulations requires good geometric fidelity, sound physics models, code verification/code software quality assurance, solution verification, validation, and uncertainty analysis 2. Each of these elements are well defined processes that may consume vast resources and should be applied with the end application in mind. This talk discusses a balanced approach to produce credible prediction for low pressure (near vacuum to 500mTorr) N2,O2 and Ar discharges that are driven by high energy radiation sources. The EMPIRE code, a new electromagnetic plasma simulation capability under development at Sandia that includes kinetic (Particle-in-Cell)plasma representation with Direct Simulation Monte Carlo (DSMC) collisions was used in this endeavor. The system studied includes photoelectrons, thermionic electrons and thermally enhanced neutral desorption from an irradiated surface along with volumetric collisional effects with a background neutral gas.   

Live

Low $T_{e}$ plasmas are widely used for thin film processing in the semiconductor industry. Modeling is an important tool for the design of these plasma systems. If one uses models to make plasmas uniform to within 1{\%} over large substrates and control ion direction within \textpm 0.1\textdegree , the models needs to sufficiently accurate. Plasma modelers are aware of the uncertainty regarding important fundamental data such as electron impact cross-sections and surface sticking coefficients. Experimental validation and calibration of plasma models are therefore critical to make them quantitatively accurate. Several examples are used to illustrate different methods for validating and refining plasma models in an industrial setting. Ideally, systematic plasma diagnostic measurements should be made on the actual plasma tool as was the case in our capacitive plasma source. $n_{e}$ and $T_{e}$ were measured using double probes in the 2 -- 162 MHz range for many gases. Another option is to validate the models under similar conditions in a research reactor. For example, diagnostic data from Ecole Polytechnique in inductively coupled halogen plasmas was used to optimize plasma models for etch applications. Processing results (e.g., etch rate, profile) are often the most easily available data. Validating plasma models using such data relies on coupling the plasma simulations to surface chemistry mechanisms, which introduces its own challenges.   

Live

This contribution addresses an important, yet sometimes overlooked, aspect of simulation/experiment benchmarking – the uncertainty caused by the interpretation of the measurement. While it is common to include error bars on measured data, these error bars typically consider only the uncertainty caused by the instrumental errors. But few diagnostic techniques are direct observations of the quantity in question and most of them rely on an analytical model for transforming the measured signal to macroscopic plasma parameters. The most common quantity used for low temperature plasma diagnostics is probably spectrally resolved light intensity. These spectral measurements are then transformed into macroscopic plasma properties (electric field, concentrations, etc) using various models. While the measured signal is already encumbered by aleatoric and epistemic uncertainty, the applied transformation brings a new sort of uncertainty connected with the uncertainty of constants entering the model that was used for the transformation. We illustrate, on a practical example, how to perform the transformation properly, accounting for the uncertainty of all the constants, and how these uncertainties manifest into the macroscopic plasma properties.\\ \\In collaboration with: Adam Obrusn\'{i}k, Masaryk University, PlasmaSolve Company; Zden\v{e}k Bonaventura, Masaryk University