Session Q60: Machine Learning of Molecules and Materials: Electronic Structure II

Machine-learning for electronic structure

Atomic-scale simulations of materials and condensed-matter systems have been transformed by the application of machine learning potentials, that facilitate and reduce greatly the computational cost for predicting their structure and (thermo)dynamics. In order to also determine functional properties, and more in general to extend further the scope of these simulations, it is desirable to develop machine-learning models that target quantities that are more intimately connected with the electronic structure -- the charge density, the electron density of states, the electronic excitations. 

I will present a few examples of this kind of models, and discuss in particular how to construct "hybrid" frameworks that combine data-driven elements with physically-motivated components. For example, I will demonstrate the use of a model of the ground-state electronic density of states to perform simulations at finite electron temperature, and the use of a minimal-basis Hamiltonian as an intermediate step in a model architecture targeting excited-state properties.    

Applying a Well-Defined Energy Density for Machine-Learned Density Functionals

The recent integration of machine learning techniques in density functional theory (DFT) has established a powerful framework for developing next generation density functionals. While robust modelling of the exchange-correlation requires a well-defined energy density, conventional training sets usually rely on global quantities. We propose the application of the local slope in the non-interacting limit of the adiabatic connection approach in DFT [1]. The talk will elucidate the methods for an efficient implementation of this quantity, with a focus on its spin-resolved components and its regularized version. Furthermore, we will highlight the potential of this strategy in paving the way for the next generation of machine-learned local dynamic hybrid functionals. Our results show a marked improvement in the prediction of observables while also maintaining computational efficiency.

References:

[1] S. Vuckovic, T. J. P. Irons, A. Savin, A. M. Teale, and P. Gori-Giorgi, “Exchange–Correlation Functionals via Local Interpolation along the Adiabatic Connection”, Journal of Chemical Theory and Computation 12, 2598–2610 (2016).   

Electronic Structures of Mesoscopic Systems: Unlocking Opportunities with Machine Learning and Orbital-Free Embedding

The pursuit of accurately predicting equilibrium and out-of-equilibrium electronic structures in mesoscopic materials is a persistent challenge. We present a paradigm that leverages density embedding to rigorously represent the electronic structure as an assembly of interacting subsystems. The subsystems are individually addressed using electronic structure solvers tailored to each subsystem. We demonstrate that state-of-the art orbital-free DFT and time-dependent DFT effectively capture the dynamical interactions between subsystems and serve as efficient electronic structure solvers for metallic and semiconducting surfaces as well as nanoparticles. Machine learning-based solvers prove highly efficient in handling subsystems of small size, significantly reducing computational complexity. We showcase specific applications of this approach in molecular liquids, materials interfaces, and solvated metal complexes.   

Pushing deep neural quantum states toward machine precision

Accessing theoretically the ground state of interacting quantum matter has remained a notorious challenge, especially for complex two-dimensional systems. Recent developments have highlighted the potential of neural quantum states to solve the quantum many-body problem by encoding the quantum many-body wave function into artificial neural networks. So far, however, this method faces the critical limitation that the training of modern large-scale deep network architectures has not yet been possible, thereby failing to capitalize on the full power of artificial neural networks. Here, we introduce a minimum-step stochastic reconfiguration (MinSR) optimization algorithm, which allows us to train unprecedentedly deep neural quantum states with up to one million parameters and 64 layers. We demonstrate our method in the paradigmatic spin-1/2 Heisenberg J1-J2 models on the square and the triangular lattices. In these systems, the deep neural networks approach different levels of machine precision on modern GPU and TPU hardware and yield significantly better variational energies compared to existing variational results. The accurate numerical results suggest the existence of gapless spin liquid in the most frustrated regime of the corresponding models. This opens up a new stage where the neural quantum state is not only applied for benchmark purposes but also helps to deepen the understanding of controversial quantum many-body systems.   

Improving neural network performance for solving quantum sign structure

Recently, using neural quantum states to solve the ground state of non-stoquastic Hamiltonian has been widely studied. However, they rely on a priori knowledge of sign structure or a pre-trained phase network. Here we propose a modified stochastic reconfiguration method, so that two separate neural networks for phase and amplitude can be trained simultaneously. Moreover, a better optimization scheme is implemented to further speed up the energy minimization process. This technique is demonstrated on the Heisenberg J1-J2 model.   

Spectral operator representations

Machine learning in atomistic materials science has grown to become a powerful tool, with most approaches focusing on atomic arrangements, typically decomposed into local atomic environments. This approach, while well-suited for machine-learned potentials, is conceptually at odds with learning complex intrinsic properties, which are often driven by spectral properties (e.g., band gaps or mobilities) that cannot be readily atomically partitioned. For such applications, methods which represent the electronic rather than the atomic structure are promising. We discuss a general framework for these spectral operator representations (SOREPs) as electronic structure descriptors which take advantage of the natural symmetries and inherent interpretability of physical models. Using this framework, we formulate a simple SOREP and apply it to the discovery of novel transparent conducting materials (TCMs) in the Materials Cloud 3D database (MC3D) using a random forest classifier. By training only on 1% (N=222) of materials in the MC3D, the model is able to correctly label more than 75% of entries in the database which meet common screening criteria for promising TCMs.   

Nonlocal neural-network distillation of many-electron density functional theory

Density functional theory (DFT) has offered a desirable balance of computational efficiency and quantitative accuracy in practical many-electron calculations for decades. Its central component, the exchange-correlation energy functional, has been approximated with increasing levels of complexity ranging from strictly local density approximations to nonlocal and orbital-dependent expressions with many empirically tuned parameters. In this work, we formulate a general way of rewriting complex density functionals using deep neural networks in a way that allows efficient computation of forces and Kohn-Sham potentials through automatic differentiation. These goals are achieved by introducing a novel class of convolutional neural network models capable of explicitly modeling functionals, as opposed to functions, while explicitly enforcing spatial symmetries. Functionals treated in this way are then called global density approximations and can be seamlessly integrated with existing DFT workflows. Tests are performed for a series of molecules and popular density functionals.   

Automatic differentiation approach for obtaining exchange-correlation functional derivatives

Obtaining explicit analytical expressions for exchange-correlation (XC) functional derivatives in density functional theory (DFT) can be tedious. Automatic differentiation methods like those implemented in the JAX framework allow us to take derivatives to the n-th order of a defined function. Here, we start from the analytical form of a known exchange-correlation energy density (ε_xc) and use automatic differentiation to get function descriptions of the first (v_xc) and second (f_xc) derivatives, known as the potential and kernel, respectively. The kernel is required for excited-state linear-response calculations in the Casida formulation of time-dependent density functional theory (TD-DFT). Simulated electronic excitation spectra using our functionals obtained from automatic differentiation show excellent agreement with the same using analytical descriptions stored within the widely-used libxc library. We scale this framework for use with meta-GGAs, orbital (OEP) and hybrid functionals, many-body dispersion functionals, as well as functionals that include electron-photon interactions (QEDFT). Newly derived functionals that only have an ε_xc expression could also make use of this framework for easier implementation in higher-order applications.   

Predicting Quantum Monte Carlo Charge Densities using Graph Neural Networks

The electronic charge density is a fundamental quantity in accurately predicting the properties of 

quantum materials. Although density functional approximations have been improving in energetics,

the progress in charge densities has been stagnant. We use quantum Monte Carlo (QMC) to calculate

charge densities as well as total and kinetic energies of over 1700 QM9 dataset molecules with

closed-shell singlet and open-shell triplet configurations. A message-passing graph neural network

designed for grid-based data was trained to predict the charge densities. We explore the viability

of reaching accurate predictions using a small QMC dataset, which we make publicly available for

future machine-learning models.   

Simple and Effective: Machine Learning-Driven Nonlocal Functionals for Orbital-Free DFT

Orbital-Free DFT holds the promise to predict the electronic structure at any temperature of mesoscopic, realistically sized systems merely requiring a computational cost that grows linearly with system size. The most accurate functionals of the kinetic energy are nonlocal with a density-dependent kernel. Unfortunately, their use is hampered by the computational complexity intrinsic in the evaluation of the kernel. In this talk, we present a useful, yet simple extension of computationally cheap nonlocal kinetic energy functionals with density-independent kernel whose kernel is evaluated by a machine learning algorithm. The resulting, ML-aided functionals show much improved applicability compared to their non-ML-aided ancestors. For example, the well-known limitation of the Wang-Teter functional in its ability to model phases of bulk Si is completely cured by its ML-aided extension employing Gaussian Process Regression. We discuss several routes for training the models involved and critically assess their performance.

References:

Shao, X., Mi, W., & Pavanello, M. (2021). Revised Huang-Carter nonlocal kinetic energy functional for semiconductors and their surfaces. Physical Review B, 104(4), 045118.

Wenhui Mi, Alessandro Genova, Michele Pavanello; Nonlocal kinetic energy functionals by functional integration. J. Chem. Phys. 14 May 2018; 148 (18): 184107.

Wang, Y. A., Govind, N., & Carter, E. A. (1999). Orbital-free kinetic-energy density functionals with a density-dependent kernel. Physical Review B, 60(24), 16350.   

Accelerating electronic structure calculations using an E(3)-equivariant neural network

The combination of deep learning and ab initio calculation has shown great promise in revolutionizing future scientific research. However, designing neural network models that effectively incorporate symmetry requirements and a priori knowledge of physical systems remains a significant challenge. Here, we present an E(3)-equivariant deep-learning framework that models the density functional theory (DFT) Hamiltonian as a function of material structure [1, 2]. The neural network respects the Euclidean symmetry of material systems and leverages the locality property of electronic matter, allowing us to achieve sub-meV level accuracy in electronic structure calculations with small-sized training structures [1-3]. Our method scales linearly with system size and is applicable to materials with up to 10⁴ atoms. Additionally, our method can be integrated with advanced computational techniques beyond the DFT level, such as hybrid functionals [4]. This not only advances the field of deep-learning method development but also opens new possibilities for materials research.   

Comparing variants of Neural Network Backflow and Hidden Fermion Determinant States

Among the variational wave function ansatz for Fermionic Hamiltonians, neural network backflow (NNBF) and hidden fermion determinant states (HFDS) are two prominent classes to provide accurate approximation for the ground states. In this work, we construct and compare a series of NNBF-type neural network states to bridge the relation between these two wave-functions showing how HFDS can be viewed as a form of NNBF. We provide both analytical and numerical results to support these conclusions considering both the limit of a large number of neurons as well as the more practical finite neuron limit.