Session T60: Machine Learning of Molecules and Materials: Materials II

Exploring equivariant models for electronic properties

In recent years, equivariant machine learning models have emerged as the preferred approach for accurately predicting the electronic properties of materials and molecules. This talk will provide an overview of various equivariant models used for different quantum chemical tasks and discuss how their architectures are adapted based on the specific requirements of each task. We will first examine PhiSNet, a neural network model designed to predict electron densities and wavefunctions. We will highlight the crucial role of equivariant coupling of higher-degree spherical harmonics in achieving accurate predictions of electronic structure and explore how the network architecture can be modified to further enhance accuracy. We will demonstrate how these accurate electronic structure approximations can be leveraged to derive other electronic properties on demand, enabling their application across a broad spectrum of quantum chemical applications. On the other hand, if we are interested in electronic properties that have a simpler structure, such as energies and forces, the coupling of higher-degree spherical harmonics may be unnecessary. For such cases, we introduce SO3krates, a lightweight model that employs only a fraction of the operations commonly found in equivariant networks while maintaining high accuracy and stability, resulting in substantial computational speedup. SO3krates can thus be used to perform long and accurate molecular dynamics simulations with a unique combination of accuracy, stability, and speed, allowing for in-depth analysis of quantum properties of matter over extended time and system size scales.   

Neural Network Backflow for ab initio quantum chemistry in second quantization

There has been recent interest in using machine learning architectures such as restricted Boltzmann machines and autoregressive neural networks for finding ground states of physical systems in second quantization. Recent work has proposed an alternative approach - the neural network backflow (NNBF) - and tested it on model systems. In this work, we focus on getting the NNBF to work with ab-initio quantum chemistry Hamiltonians. We explore and improve various different optimization techniques including deterministic approaches and benchmark the efficacy of NNBF as we increase the system size as well as approach the complete basis set limit. This research paves the way for more efficient ab-initio simulations.   

Avoiding a reproducibility crisis in deep learning for surrogate potentials: How massively parallel programming, millions of training steps, and numerics combine to create non-determinism in models and what this means for the simulated physics

Deep learning has recently provided ground-breaking results in scientific areas, including for molecular dynamics simulations.  For many researchers, the possibility of training an interatomic potential on first principles data, then performing molecular dynamics with quantum mechanical accuracy at the speeds of empirical models, represents a fantasy realized. Deep learning frameworks such as TensorFlow and PyTorch rely heavily on massively-parallel speedups from graphics processing units (GPUs); data-distributed parallel training across multiple GPUs is also common. Recently, however, deep learning training has been discovered to produce significantly different models when trained repeatedly using identical algorithms, hyperparameters and input data, even with all random seeds set. This non-determinism is due to floating-point non-associativity coupled with atomic operations, when performing iterative calculations millions to billions of times. Non-reproducibility of model training has interfered with approvals for safety-critical autonomous vehicle software and medical diagnostics. Similar repercussions are expected for scientific applications. Here we study the effects of non-determinism on the models produced for deep neural network potentials for molecular dynamics simulations, including how this model variability translates to variability in the description of the physical phase space and associated observables, and offer tips and best practices for achieving correctness and reproducibility within this new paradigm.   

Free energy simulations with machine learning-based forcefields for prediction of thermodynamic properties of molten salts

The high specific heat capacity and low volatility of molten salts makes them excellent candidates for many high temperature applications, such as molten salt nuclear reactors and concentrated solar power plants. However, the design and optimization of molten salts and their mixtures for targeted applications is largely hindered without cost-effective approaches for quickly characterizing candidates from the vast compositional space available. For example, the ability to computationally predict chemical potentials would grant insight into many chemical properties that are pertinent to molten salt applications, such as salt basicity, solubilities, phase boundaries, and redox potentials. However, the high computational cost of quantum chemical methods, such as density functional theory (DFT), limits the use of advanced techniques, such as thermodynamic integration (TI), for rapidly computing chemical potentials. To address this issue, we have utilized machine learning-based forcefields (MLFFs) that are trained to accurately reproduce DFT energies, forces, and stresses at a fraction of the cost, thereby enabling TI calculations. In this talk, we will describe several methods we have tested for computing the excess chemical potential of molten lithium chloride using TI and MLFFs. We will employ the most robust methodology for predicting redox potentials of several transition metals in molten lithium chloride and mixing free energies in the LiCl-CsCl system.  This work showcases the key advantages and unique challenges of leveraging MLFFs in thermodynamic property predictions and lays the foundation for the rapid characterization of molten salt systems.   

JARVIS-Leaderboard: Large Scale Benchmark of Materials Design Methods

Lack of rigorous reproducibility and validation are major hurdles for scientific development across many fields. Materials science in particular encompasses a variety of experimental and theoretical approaches that require careful benchmarking. Leaderboard efforts have been developed previously to mitigate these issues. However, a comprehensive comparison and benchmarking on an integrated platform with multiple data modalities with both perfect and defect materials data is still lacking. This work introduces JARVIS-Leaderboard, an open-source and community-driven platform that facilitates benchmarking and enhances reproducibility. The platform allows users to set up benchmarks with custom tasks and enables contributions in the form of dataset, code, and meta-data submissions. We cover the following materials design categories: Artificial Intelligence (AI), Electronic Structure (ES), Force-fields (FF), Quantum Computation (QC) and Experiments (EXP). For AI, we cover several types of input data, including atomic structures, atomistic images, spectra, and text. For ES, we consider multiple ES approaches, software packages, pseudopotentials, materials, and properties, comparing results to experiment. For FF, we compare multiple approaches for material property predictions. For QC, we benchmark Hamiltonian simulations using various quantum algorithms and circuits. Finally, for experiments, we use the inter-laboratory approach to establish benchmarks. There are 1281 contributions to 274 benchmarks using 152 methods with more than 8 million data-points, and the leaderboard is continuously expanding. The JARVIS-Leaderboard is available at the website: https://pages.nist.gov/jarvis_leaderboard/   

First-principles study of THz dielectric properties of liquid molecules with a machine learning model for dipole moments

The dielectric response of materials in the THz region has been studied extensively in recent years due to improvements in experimental techniques and increased industrial interest. Theoretically, the dielectric response is calculated from dipole moments collected along a molecular dynamics trajectory. Therefore, it is necessary not only to get accurate trajectories but also to calculate dipole moments precisely footnote{C. C. Wang, J. Y. Tan, and L. H. Liu, AIP Advances 7, 035115 (2017).}. Recently, machine learning of molecular dipole moments has been studied using the centroid of Wannier functions calculated from first principlesfootnote{ A. Krishnamoorthy, K. Nomura, N. Baradwaj et al., Phys. Rev. Lett. 126, 216403 (2021).}$^{,}$footnote{ L. Zhang, M. Chen, X. Wu et al., Phys. Rev. B 102, 041121 (2020).}. We have constructed a versatile machine learning model of dipole moments applicable to molecular liquids. We assigned Wannier functions to chemical bonds between atoms and used deep neural networks to predict the position of the Wannier function for each bond, which is applicable to complex materials. We applied our method to calculating the dielectric function of liquid alcohols and obtained results that agreed well with the experimental ones.   

Machine learning molecular conformational energies using semi-local density fingerprints

With the potential to sidestep the steep cost associated with high-level quantum-chemical calculations, machine learning (ML) has become an increasingly more viable approach in the field of theoretical and computational chemistry/physics over the past decade. In this work, we describe a novel molecular descriptor that goes beyond structural data and incorporates the wealth of information contained in semi-local descriptors of the electron density (i.e., ρ(r) and ▽ρ(r))—the quantum-mechanical objects at the very heart of density functional theory (DFT). The proposed “semi-local density fingerprint” (SLDF) molecular descriptor transforms the most energetically-relevant information in ρ(r) into a unique and compact (system-size-independent) representation for each molecule. By construction, SLDFs are global molecular descriptors that are atomic-species independent, invariant to translations, rotations, and permutations, and account for molecular symmetry. In a series of proof-of-principle tests, we demonstrate that SLDF-based ML models are able to predict molecular conformational energies and complex potential energy surfaces with high fidelity, even when the training does not include the test molecule.   

Spectroscopy of two-dimensional interacting lattice electrons using symmetry-awareneural backflow transformations

Neural networks have shown to be a powerful tool to represent many-body states, including for

fermionic systems. In this paper, we introduce a framework for embedding lattice symmetries in

Slater-Backflow-Jastrow wavefunction ansatze, and demonstrate how our model allows us to map the

ground state and low-lying excited states. To capture the Hamiltonian symmetries, we introduce

group-equivariant backflow transformations. We benchmark our ansatz on the t-V model on a

squared lattice and find that it significantly decreases the relative error when searching for ground

states, and accurately accesses low-lying excited states.In addition, our ansatz is able to compute

other observables such as the two-point-density correlation function and the structure factor in

order to detect the phase transiton critical point. Finally, we quantify the variational accuracy of

the model with the V-score.   

Using Machine Learning to Predict the Adsorption Properties of Thiophene (C4H4S)

We present a machine learning (ML) study of the adsorption of thiophene (C₄H₄S) on various single metal and bimetallic (100) transition metal surfaces.  We employ the Hierarchically Interacting Particle Neural Network (HIP-NN) to make our predictions.  HIP-NN, a deep neural network, uses both local atomic density and pairwise atom information to make predictions.   For our training dataset we use a database of over 2400 thiophene adsorption calculations generated with Quantum Espresso using the BEEF-vdW density functional theory (DFT) functional.    These configurations have the thiophene molecule adsorbed in a parallel absorption configuration with the molecule centered over a hollow site with the sulfur atom near an atop site. This adsorption site has been shown in previous studies to be the most energetically favorable adsorption site.  Overall, we make predictions, using ML, of adsorption energies, adsorption heights, buckling of the surface, and charge transfer from the surface to the S atom to the accuracy of the DFT calculations, and try to correlate these properties, namely charge transfer and adsorption energy, to the hydrodesulfurization rates of the sulfurized version of these surfaces.    

Machine Learned Interatomic Potentials to Predict Solvatochromic and Stokes Shifts

Accurate simulations of the excited state dynamics of chromophores in complex environments are a prerequisite to understanding important properties such as photostability, relaxation pathways, and excited state lifetimes. An atomistic understanding of these properties can aid in the design and study of useful chemical compounds such as novel sunscreen candidates. Conflicting requirements set by the need to follow individual trajectories over long timescales, to sample over the ensemble of solvent configurations, and to use a high level of theory to obtain good chemical accuracy puts such methods out of the reach of DFT or QC methods alone. Machine learning allows us to accelerate dynamics simulations of chromophores, including Methyl Anthranilate in a variety of solvent environments, for ground and excited states, while maintaining the accuracy of DFT and higher-level methods. Our workflow to predict peak positions and widths, and solvatochromic and stokes’ shifts, is based on ESTEEM: Explicit Solvent Toolkit for Electronic Excitations of Molecules [1].

[1] esteem.readthedocs.io   

Electronic stopping power predictions from machine learning

We aim to develop an affordable computational approach that provides the electronic stoppingpower for arbitrary trajectories of ions impacting a target material with an accuracy comparable to that of modern quantum mechanical first-principles simulations. Currently, real-time time-dependent density functional theory can accomplish this in reasonable agreement with experiment. However, the computational cost of this method is high which limits the number of trajectories and host material atomic geometries that can be studied. This prevents a routine integration of electronic-stopping power, e.g. in the molecular dynamics simulation of radiation damage cascades. We use cutting-edge descriptors of atomic geometries to train modern machine-learning models on data from real-time time-dependent density functional theory. We find very low error bars and very high accuracy at million-fold reduced computational cost of the trained model for proton irradiated aluminum. We also are able to predict velocity dependent electronic stopping and entire Bragg peak simulations with our models. In this presentation we discuss our framework in detail as well as its broad applicability in the particle-radiation community, including target materials with complex atomic geometry or low-dimensional materials.   

Predicting Properties of van der Waals Magnets using Graph Neural Networks

We present a study of two dimensional (2D) magnetic materials using state-of-the-art machine learning models that use a graph-theory framework.  Representing materials as graphs allows us to better learn structure-property relationships by leveraging both the chemical properties of the constituent atoms and the connectivity between those atoms. These models are capable of predicting both structure-level (graph-wise) and atom-level (node-wise) features. By simultaneously making predictions on both types of features, we may force our model to learn relationships between local and global properties. This constraint guides the model to more accurately capture the underlying physical interactions. In particular, we train a graph neural network that uses the Atomistic Line Graph Neural Network (ALIGNN) architecture. We train the ALIGNN model on data comprising DFT calculated local and global magnetic moments of 314 2D structures of the form CrAⁱⁱBⁱBⁱⁱX₆ based on monolayer Cr₂Ge₂Te₆. By learning the relationships between local and global magnetic properties, we demonstrate an improvement over models that are only trained on global magnetic properties.   

Optimizing machine learning electronic structure methods based on the one-electron reduced density matrix

Regression methods can be employed to "learn" the electronic structure of molecules and materials, e.g. by learning the electron density [1] [2], the spectral density [3], or the one-body reduced density matrix (1-rdm) [4]. We present a comprehensive analysis of the regression model used for learning the map linking the external potential to the 1-rdm. We span kernel ridge regressions (KRR), as well as deep learning utilizing a common set of training data. We employ cross-validation techniques to refine hyper-parameters of each KRR model. We find that KRR with an optimized RBF kernel (a Gaussian kernel) generally outperforms other models and provides us with 1-rdms that do not require secondary learning steps to reach self-consistent field quality. Our work extends the applicability and robustness of the current state-of-the-art models [4].