Session Q53: AI and Statistical/Thermal Physics

Self-assembly of electronic materials and the power of machine learning

There are many problems at the forefront of materials chemistry whose solution is stymied by its inherent complexity. Such problems are characterized by a rich landscape of parameters and processing variables that is combinatorially too large for either an experimental or a computational approach to solve through an exhaustive search. In such cases, the usual approach is an Edisonian trial-and-error approach, which inevitably leaves areas of parameter space unexplored. The problems that we have explored are also characterized by a scarcity of data, since the data are expensive to acquire, both experimentally and computationally. This makes it an ideal candidate to solve using a Bayesian optimization (BayesOpt) approach.

We have developed and used a Bayesian approach to study several such problems in materials chemistry. This talk will discuss the most mature test case in which we looked at “solvent engineering” to optimize the process by which some novel solar cells are fabricated. We developed a BayesOpt algorithm that is “physics-informed”

through its belief choice. We use that algorithm to study how to optimize the choice of solvent or additive to produce high quality thin films of lead-based hybrid organic-inorganic perovskites. These “perovskites” all exhibit a crystalline habit with the formula ABX3, where A and B are cations and X is an anion. Here, we chose B to be lead and X to be halides, and we allow A-site cations to be chosen among a small group of options. Using BayesOpt, we identified the best solvent out of a list of around 12 solvents, and also how to optimize the best mixed-halide system to solubilize the lead salt.

To validate these predictions, we used the Bayesian-generated ranked list of “best performers” to identify when a crystalline “intermediate” is likely to form, which can adversely affect thin film quality. We confirmed our ability to determine whether a precursor forms in six cases (three positive for crystalline intermediates and three negative for intermediates). To do so, we were fortunate to have XRD experiments performed by Lara Estroff’s group at Cornell that confirm our predictions.

    

How deep neural networks learn thermal phase transitions

Machine-learning methods have successfully been used to identify phase transitions from data. Neural network (NN)-based approaches are particularly appealing due to the ability of NNs to learn arbitrary functions. However, the larger an NN, the more computational resources are needed to train it, and the more difficult it is to understand its decision making. We derive analytical expressions for the optimal output of three popular NN-based methods for detecting phase transitions which rely on solving classification and regression tasks using supervised learning at their core [1]. This result corresponds to the output when using optimal predictive models, such as sufficiently large NNs after ideal training. Our analysis reveals that high-capacity neural networks ultimately gauge changes in the energy distributions as a function of temperature when used to detect thermal phase transitions. In contrast, low-capacity neural networks seem to learn order parameters, i.e., recognize prevalent patterns or orderings. Our theoretical findings are supported by analyzing data from numerical simulations of classical spin systems.   

Machine learning phases of matter: Scalability and limitations

We present a scalable machine learning (ML) framework for distinguishing phases and identifying phase transitions in many-body systems. Scalability and transferability are central to the unprecedented computational efficiency of ML methods.  In general, linear-scaling computation can be achieved through the divide and conquer approach, and the locality of physical properties is key to partitioning the system into sub-domains that can be solved individually. Our proposed ML framework for phase classification is similar in spirit to the Behle-Parrinello approach widely employed in ML-based quantum molecular dynamics simulations. We discuss the limitations of this approach and the important role of the correlation length. The prototypical two-dimensional Ising model is used to demonstrate the proposed framework. Implications for practical applications are also discussed.    

Learning Together: Training Interatomic Potentials to Multiple Datasets

The development of machine learning interatomic potentials has led to an abundance of datasets containing quantum mechanical calculations for molecular and material systems. However, using the information from different datasets together remains a challenge due to the varying levels of theory employed. In this talk, we show that techniques can be used to fit an interatomic potential to multiple organic molecule datasets and that this yields ML potentials with improved accuracies for a variety of tasks.   

Local force field of thermally displaced atoms in unstable bcc iron from machine learning

A dataset of energy versus atomic thermal displacements was created from density functional theory molecular dynamics simulations of non-spin-polarized body-centered cubic iron at pressures centered at 7 GPa and temperatures centered at 1700K, and Gaussian process regression was used to make predictions based on the similarity between atomic displacements as determined by a graph kernel. Force versus displacement relationships were computed at randomly selected timesteps of the simulations for all atoms in the supercell in the directions of their first-, second-, and third- nearest-neighbors. The atoms experience a generally restorative force in the direction of their first- and third-nearest neighbors, but are unstable in the direction of their second-nearest neighbors. The predicted dynamics are consistent with a martensitic phase transition to face-centered cubic, which is the thermodynamically stable phase of iron at the investigated temperature and pressure conditions.   

CHGNet: Pretrained Neural Network Potential for Fast and Accurate Charge-constrained Molecular Dynamics

Molecular dynamics (MD) simulation coupled on systems with complex electron interactions remains one of the biggest challenges for atomistic modeling. While classical force fields often fail to describe the electronic coupling with ionic rearrangements, the more accurate spin-polarized ab initio molecular dynamics (AIMD) suffer from the computational complexity that prevents long-time and large-scale simulation, which are essential to study ion migrations and phase transformations.

In this work, we present the Crystal Hamiltonian Graph Neural Network (CHGNet) as a novel approach that uses a graph neural network (GNN) based force field to model a universal potential energy surface that can describe both atoms and electrons. CHGNet is pretrained on a large Materials Project Trajectory (MPtrj) Dataset, which consists of over 1 million inorganic structures from over 10 years of density functional theory (DFT) static and relaxation trajectories at the Materials Project. We demonstrate the performance of CHGNet molecular dynamics in Li-ion solid-state electrolyte and phase transformation in Li-ion cathode materials.

    

Neural-network-based interatomic potential: A case study on lithium

Advancements in neural-network-based force fields have led to the predictions of materials for applications in widespread applications. In this work, we will show a general scheme that can be used to develop a neural-network-based interatomic potential using our in-house developed python atom-centered machine learning force field package (PyAMFF) with GPU capabilities. Using an example of lithium, we will show a force field can be developed using neural networks: (a) Data Collection step: atomic positions, energies, and forces from density functional theory (DFT) calculations for different lithium systems; (b) Fingerprint Selection step: an automated Behler-Parrinello representation selection for a dataset using radial and angular distribution function; (c) Training Dataset Generation step: selection criteria in fingerprint space to reduce the size of DFT dataset for neural network training; (d) Model Training step: analyzing the effect of neural network size and fingerprint selection on model accuracy; (e) Model Performance step: rigorous testing of neural-network-based force field on rare event searches using Adaptive Kinetic Monte Carlo and global optimization of lithium clusters using basin hopping. This force field will help in answering questions related to kinetics of lithium deposition on lithium metal surfaces at experimental timescales for battery applications.   

On-the-fly Machine Learning-Accelerated Geometry Optimization: Theoretical Screening of a Single Atom Alloy for CO2 Electroreduction Reaction

Force-fields based upon machine learning (ML) methods are being adopted for the design of catalysts due to their high efficiency and first-principles level accuracy. Such efficiency and accuracy are important particularly for high-throughput computations, which are key to the computational design of new catalysts. Among various types of catalysts, single-atom alloy (SAA) catalysts, which substitute a single atom from the surface for a different type of atom, have exhibited outstanding selectivity and activity due to their unique geometry. There have been a number of theoretical studies to find optimal combinations of SAA for catalytic reactions but screening all possible combinations remains intractable due to the computational expense. Here, we show a mathematical modeling package of training potential energy surfaces using artificial neural networks and applying the machine-learned models to accelerate geometry optimization process. On-the-fly generated machine learning force-fields are used as the basis for a separate optimization to reach a local minima or saddle point on the surrogate PES, which requires less computational effort than the same steps using DFT. Iteration between ML-optimization and DFT calculations will reduce the overall number of DFT force calls required for optimizations, while retaining DFT accuracy for high-throughput screening.   

Investigating the influence of local composition on properties in complex alloys using machine learned interatomic potentials

Complex concentrated alloys (CCAs) contain high concentrations of three or more metallic elements which mix exceptionally well, down to the atomic scale. In contrast to conventional alloy properties, CCA properties are highly sensitive to local chemistry and the specific scale of property measurement. For a multitude of reasons, molecular dynamics (MD) stands out as uniquely suited to probe this distinctive scale-sensitivity. However, there are two major, interrelated challenges to tackle before MD can make real inroads into this area. The first is the development of transferrable interatomic potentials that can accurately model CCAs through widely varying chemical environments. Highly accurate machine-learned interatomic potentials (MLIAPs) hold enormous promise as a solution, but training and validating CCA MLIAPs is still a nascent field. The second challenge is that, even with excellent CCA MLIAPs, little progress can be made without ways to define, track, and compare the ‘local’ alloy compositions contained in a CCA sample. Relying exclusively on a traditional ‘global’ alloy composition label (such as ‘equiatomic’) to describe properties hinders understanding of the varied nanoscale chemical interactions that characterize CCAs.

In this presentation, we will discuss our group’s progress in addressing both challenges. We will show successful approaches to developing robust, multi-compositional CCA MLIAPs using the spectral neighbor analysis potential (SNAP) formalism. Using SNAP models of the MoNbTaTi(W) refractory CCA family, we will demonstrate how metrics built in an alloy’s composition space can be used to connect local and global CCA properties in already-trained MLIAPs and be used to make data-driven decisions in the training of new ones. Finally, we will discuss the potential these advances have to influence alloy characterization, design, and property prediction.  Progress in defining training robustness in this space translates broadly to many supervised machine learning tasks.   

Nanoparticle Heterogeneous Catalysis Dynamics Simulations with Machine Learned Force Fields

Quantitative understanding and control of interfacial reactions between the gas-phase and solid surfaces are crucial for improving numerous catalysis and energy conversion systems. Examples of these interfacial phenomena include H₂ splitting and CO adsorption on nanoparticles, both of which are important industrial processes and lead to markedly different particle behaviors. These disparate particle responses cannot be resolved under the spatial and temporal resolutions of current experimental techniques, lending this problem to be solved by molecular dynamics simulations. We train a collection of robust, Bayesian machine learned force fields (MLFFs) using the FLARE on-the-fly active learning framework implemented using Gaussian Process regression. Molecular dynamics (MD) simulations are used to sample atomic configurations and density functional theory is only called upon when the Bayesian uncertainty exceeds a threshold. This workflow yields both acceleration in time-to-solution and an increase in computational efficiency relative to ab initio MD. The resulting MLFFs retain first principles accuracy, are fast, and are uncertainty-aware. Following a rigorous validation scheme, through comparison of the dynamic evolution of these particles and bulk systems to available x-ray data (i.e., extended x-ray absorption fine spectra) and static benchmarks, long timescale MD simulations for freestanding metal nanoparticle systems (e.g., Pt, Au, PdAu, and CuPt) are performed. In addition to the bare particles, reaction mechanisms under gaseous exposure (e.g., H₂ and CO) are also investigated. These MLFFs allow for the simultaneous study of atomistic mechanisms occurring on these nanoparticles under exposure to reactive atmospheres and the evolution of their structural morphologies.   

Melting and Phase Separation of SiC from Large-scale Machine Learning Molecular Dynamics

Incongruent melting has been reported by many experimental studies with the observation that the SiC decomposes into liquid silicon and solid carbon upon heating. However, contradictory results have also been reported that SiC melts congruently with no decomposition. Ab initio MD provides evidence of carbon clustering during the melting process but did not capture the phase separation limited by the small supercell. Empirical potentials give good approximations of the melting points but all of them have limited ability to describe other phases and thus cannot capture the decomposition in the MD simulation. In this work, we use Bayesian active learning to train a machine learning potential for SiC at a wide range of temperatures and pressures, and observe the phase decomposition process into Si and C-rich phases in both the active learning and the large-scale MD stages. Our results provide a microscopic validation of the incongruent melting hypothesis, resolving the disagreement among experiments of the SiC melting behavior. The occurrence of decomposition also indicates that amorphous SiC can not be produced by melting and quenching but only with irradiation, consistent with existing experimental approaches of the amorphization of SiC.   

ML models for partition functions: from the prediction of thermodynamic properties to the exploration of transition pathways

We discuss our recent work on the development of machine learning (ML) models for the prediction of partition functions. To this end, we carry out computationally-intensive flat histogram Monte Carlo simulations, based on a Wang-Landau sampling scheme, to obtain the partition function for atomic and molecular systems. The simulation results for the partition functions are then gathered in datasets that allow us to train and validate ML models. This, in turn, leads us to build artificial neural networks models for the prediction of partition functions for a broad range of conditions. We demonstrate the accuracy and reliability of the ML models by showing the ability of the ML-partition function to predict thermodynamic properties for single component-systems, mixtures, confined fluids. Furthermore, we show that the ML-partition function can be leveraged to build reaction coordinates for the exploration of phase transition pathways.   

Diffeomorphisms invariance is a proxy of performance in deep neural networks

Understanding why deep nets can classify data in large dimensions remains a challenge. It has been proposed that they do so by becoming stable to diffeomorphisms, yet existing empirical measurements support that it is often not the case. We revisit this question by defining a maximum-entropy distribution on diffeomorphisms, that allows to study typical diffeomorphisms of a given norm. We confirm that stability toward diffeomorphisms does not strongly correlate to performance on benchmark data sets of images. By contrast, we find that the stability toward diffeomorphisms relative to that of generic transformations R_f correlates remarkably with the test error ε_t. It is of order unity at initialization but decreases by several decades during training for state-of-the-art architectures. For CIFAR10 and 15 known architectures, we find ε_t ∼ 0.2 √R_f, suggesting that obtaining a small R_f is important to achieve good performance.