Session T18: Advances in AI/ML-Driven Sampling for Atomistic Simulations

Walking the surfaces with AI-powered MD simulations

Peptides and enzymes have emerged as incredibly promising building blocks for materials with tunable properties. Self-assembly-driven peptide materials have a broad range of applications such as drug delivery, sensing, and separations. Enzymes too have several biotechnological applications including sensing and degrading plastics. In all these applications, it is crucial to understand the behavior of these peptides/enzymes as a function of their sequences (i.e., composition) and solution conditions such as temperature, pressure, additives, and surfaces. In our work, we explore these aspects using molecular simulations. We use various computational techniques such as molecular dynamics, network analysis, path sampling, and machine-learning enhanced sampling. In my talk, I will talk about our recent work on developing ML-enhanced methods to study peptides on interfaces. We will illustrate that our methodology has the potential to overcome the challenge of conformation sampling on interfaces. This enables us to study large conformational changes that could occur when proteins interact with surfaces. We will demonstrate the power of such approaches through studies of proteins on polymeric surfaces motivated by biodegradable polymer design applications.   

Enhanced Sampling using Birth-Death Algorithm

Molecular dynamics simulations face challenges to effectively sample rare event scenarios such as transitions between metastable states that are separated by high-energy barriers, all within a specific time frame. One approach to address this issue is through collective variable -based enhanced sampling methods. These methods involve multiple walkers with the same bias potential to collectively sample the free energy landscape, thereby reducing convergence time. However, when walkers become correlated, it becomes crucial to implement a population control strategy to enhance their performance. In this talk, we introduce one such population control scheme for running multiple walkers in parallel, where we enhance the molecular sampling strategy by augmenting with a birth/death process¹. This process periodically eliminates and duplicates walkers based on a Fokker-Planck Birth-Death equation.

References

1.        Pampel, B., Holbach, S., Hartung, L. & Valsson, O. Sampling rare event energy landscapes via birth-death augmented dynamics. Phys. Rev. E 107, 24141 (2023).   

Stochastic Resetting for Enhanced Sampling

We present a new approach for enhanced sampling of Molecular Dynamics (MD) simulations using stochastic resetting (SR).

MD simulations is a powerful tool, used for the study of physical and chemical systems at the microscopic level. However, due to their atomic resolution, MD simulations are limited to processes shorter than a few microseconds. Longer processes, such as protein folding, or crystal nucleation and growth, cannot be sampled by standard MD simulations.

SR is the procedure of stopping random processes and restarting them, resampling independent and identically distributed initial conditions. It was shown to expedite different kinds of stochastic processes, ranging from queuing systems to diffusion of colloidal particles. Here, we employ it to enhance MD simulations for the first time, leading to speedups of up to an order of magnitude. We also present an inference procedure, to obtain the unbiased kinetics from simulations with SR.

Next, we demonstrate that SR can be combined with existing enhanced sampling methods, such as Metadynamics (MetaD). For a simple model system, we show that this combination may lead to higher speedups than either approach independently. In another model, we show that restarting MetaD simulations with sub-optimal collective variables (CVs) gives comparable accelerations to using the optimal CV, suggesting resetting can be an easy alternative to improving CVs. We apply the combined approach to alanine tetrapeptide, showing that SR expedites simulations with bad CVs by a factor of 150. Lastly, we present an inference procedure of unbiased kinetics for the combined MetaD and SR method.   

Manifold Learning of Collective Variables for Enhanced Sampling Simulations

Among the main challenges in atomistic simulations of biomolecular systems is the so-called sampling problem or rare event problem, where proper sampling of energy landscapes is impeded by high kinetic barriers hindering transitions between metastable states on typical simulation time scales. Numerous enhanced sampling methods have been developed to address this problem and more efficiently sample rave event systems. Many such enhanced sampling methods work by identifying a few slow degrees of freedom, termed collective variables (CVs), and enhancing the sampling along these CVs. However, selecting CVs to analyze and drive the sampling is not trivial and often relies on chemical intuition. Machine learning (ML) methods, in particular dimensionality reduction or manifold learning methods, provide a possible solution to this issue.

Here, we present a manifold learning method called multiscale reweighted stochastic embedding (MRSE) for automatically constructing CVs to represent and drive the sampling of free energy landscapes in enhanced sampling simulations. The technique automatically finds CVs by learning a low-dimensional embedding of the high-dimensional feature space to the latent space via a deep neural network. Our work builds upon the popular t-distributed stochastic neighbor embedding (t-SNE) approach and addresses known limitations of t-SNE.

We introduce several new aspects to stochastic neighbor embedding algorithms that make MRSE suitable for enhanced sampling simulations, including: (1) a well-tempered landmark selection scheme; (2) a multiscale representation of the high-dimensional feature space; and (3) a reweighting scheme to account for biased training data. We show the performance of MRSE by applying it to several molecular systems. Furthermore, we present a theoretical motivation behind our reweighting scheme and show how it can be generalized to other manifold learning techniques.   

Efficient Sampling for Structure Search Using VAE-Organized Latent Spaces and Genetic Algorithms

Structure search through evolutionary algorithm methods is well established for novel materials discovery in a specific chemical system. The evolutionary search typically occurs in the structure space, which is not necessarily organized according to desired properties such as stability or band gap. To address this, we have developed an extension to our in-house multi-objective evolutionary algorithm package, FANTASTX (Fully Automated Nanoscale To Atomistic Structure from Theory and eXperiments). The updated framework initially performs the genetic algorithm-based structure search and uses the generated data to train a variational autoencoder (VAE). We use an auxiliary network to organize the VAE latent space according to a desired property, which can be rapidly predicted by an independent, trained surrogate model. We sample new structures from the latent space and evaluate them using density functional theory (DFT) or force fields. In this talk, we will discuss the performance of the generative and evolutionary synergistic framework based on the tests performed on the CdTe system using force fields and the IrOx system using DFT calculations.   

Complex local environments classification of shape particles through shape-symmetry encoded data augmentation.

Detecting and analyzing the local environment is crucial for investigating the dynamical processes of crystal nucleation and shape colloidal particle self-assembly. However, the application of machine learning to self-assembly on shape particles is still underexplored. To address this gap, we propose a simple, physics-agnostic, yet powerful approach that involves training a multilayer perceptron (MLP) as a local environment classifier for shape particles, using input features such as particle distances and orientations. Our MLP classifier is trained in a supervised manner with a shape symmetry-encoded data augmentation technique without the need for any conventional roto-translations invariant symmetry functions. We evaluate the performance of our classifiers on four different scenarios involving self-assembly of hard cubes, 2-dimensional and 3-dimensional patchy shape particle systems, hexagonal bipyramids with varying aspect ratios, and truncated shapes with different degrees of truncation. The proposed training process and data augmentation technique are both straightforward and flexible, enabling easy application of the classifier to other processes involving particle orientations. Our work thus presents a valuable tool for investigating self-assembly processes on shape particles, with potential applications in structure identification of molecular or coarse-grained systems.   

Learning all-atom molecular reactions using data-driven approaches

Molecular reactivity spans extended length and time scales, making them costly to simulate using accurate density functional theory (DFT) approaches or limited in chemical transferability/scope using reactive force fields. In this work, we describe how simulations of liquid structure and dynamics of organic molecules undergoing thermal decomposition reactions can be achieved using hybrid DFT, active learning [1], and machine learning force fields [2].

 

Active learning reduces the overall computational cost of hybrid DFT by employing a fast, uncertainty-aware force field to collect only “sufficiently”-uncorrelated DFT frames. In addition, high-temperature configurations with radicals or free gasses, which are potential decomposition products, can be collected on-the-fly. The approach is useful in situations where classical force fields are either unavailable or lacking in expressivity.

When training data from active learning are fed into a data-efficient equivariant neural network, molecular decomposition and reaction pathways can be traced with first-principles, all-atom resolution by identifying reaction pathways to product formation. We apply the approach to study the thermal decomposition of a “green” solvent used in battery recycling and validate our results against experimental characterization.   

Exploring transferability of machine learning interatomic potentials for reactive chemistry

The development of machine learning interatomic potentials (MLIP) has resulted in the ability to model high-quality potential energy surfaces with near ab initio level of accuracy at low computational cost. However, just like other machine learning models, MLIP faces challenges when it comes to transferability, specifically to systems of chemical space beyond its training. Here we explore sampling techniques that can allow MLIP such as ANI and NequIP to obtain transferability beyond its training data to achieve accurate bond dissociation across chemical space.   

Active Learning of Diffusion Pathways for Machine-Learned Interatomic Potentials

In chemically complex systems such as MPEAs, the number of elemental permutations involved in the local environment of even simple diffusion pathways, such as vacancy and interstitial, grows factorially with the number of alloying elements. As such, it becomes computationally intractable to employ ab initio methods to study diffusion kinetics with representative size-scale structures or through pathway enumeration. Here we show that machine-learned interatomic potentials (MLIPs) offer a compelling solution since they can scale to representative sizes and can be explicitly trained on diffusion pathways to reproduce them accurately. To that end, we present a workflow that includes different diffusion pathway sampling strategies used in conjunction with active learning to generate MLIP training data for kinetic modelling. We report on best practices for data-efficient training using as our benchmark system, NbTiZr alloys with oxygen interstitials, which recently have shown to exhibit complex passivation dynamics.   

Modeling reaction-diffusion in the liquid-phase heterogeneous catalysis using machine-learned force field.

Modeling liquid-phase heterogeneous catalysis is challenging because it involves disparate time and length scales. Typically, density functional theory is employed for modeling chemical transformations, utilizing an idealized catalyst model. On the other hand, the diffusion process, characterized by considerably longer time scales, is simulated using pair-wise additive force fields. Nonetheless, considering the pivotal role of solvation and confinement in liquid-phase zeolitic catalysis, it becomes imperative to achieve near-quantum mechanical accuracy when modeling the entire catalytic process. Recent advancements in machine learning (ML) promise to bridge this gap. In this presentation, we will share our recent endeavors in modeling liquid-phase reactions catalyzed by zeolites, shedding light on our achievements and challenges.   

Charge-dependent atomic cluster expansions

Machine-learned interatomic potentials (ML-IAPs) are valuable tools for the computational modeling of materials and chemical systems. In many cases they offer highly accurate alternatives to many empirical interatomic potentials. Many of the ML-IAPs rely on the use of descriptors that encode the local chemical bonding environment about a central atom. This local description of atomic interactions restricts many ML-IAPs to systems where long-range effects are negligible, where long-range effects are sufficiently screened, or where only small amounts of charge transfer occur. Long-range coulomb interactions and charge transfer are not insignificant when considering phenomena such as corrosion, oxidation, and related atomic systems. Many recent advances in ML-IAPs, such as message-passing networks and charge-dependent descriptors, help alleviate these deficiencies, however these new developments have their own limitations. Their utility is often limited by the methods for determining atomic charges. Like empirical potentials, methods for charge relaxation are often inaccurate and/or expensive. Using newly developed charge-dependent atomic cluster expansion (ACE) descriptors and shadow molecular dynamics-inspired charge relaxation schemes, we help address some of these challenges with charge-dependent ML-IAPs. In this work, we explore the benefits of using charge-dependent ACE ML-IAPs and how they can help correct spurious behavior typically encountered in dynamic-charge molecular dynamics simulations.   

Rapidly converging cluster expansions by transfer learning from empirical potentials

Cluster expansion (CE) is an effective and widely used method for mapping the potential energy landscape of multicomponent crystalline systems. It enables a systematic and efficient exploration of multidimensional configurational spaces. Despite considerable advances in CE methodologies [1], there remain challenges associated with the sampling of lattice configurations when applied to systems with low symmetry or a high number of chemical components (e.g., high-entropy alloys). In this work, we address these challenges by employing active learning to reduce the number of first-principles calculations in a large training set. Our approach utilizes Bayesian analysis to leverage empirical priors from interatomic potentials, enabling the identification of the most relevant configurations within the training set. To attain this objective, the initial step involves calculating the energy of each structure using empirical potentials, such as COMB3 and EAM, to create a Gaussian distribution. We then enhance this distribution by calculating the energies of most relevant structures using density-functional theory until convergence of the CE enthalpies is achieved. The effectiveness of the approach is tested on the Pt–Ni alloy system, yielding a 2.5-fold decrease in the number of calculations required for constructing a cluster expansion while ensuring robust convergence of the cluster-expanded Hamiltonian with limited statistical fluctuations.

[1] Mueller, Ceder. Phys. Rev. B, 80:024103, 2009.    

Accelerated Predictions of Charge Density Evolution in MD simulations Using Machine Learning

Reactive force field-based molecular dynamics simulations can be utilized to build an understanding of the effects of a salt brine on the corrosion-based processes experienced by an underlying metallic substrate. However, due to the time and memory intensive nature of performing these simulations, forecasting the long-term behavior at the interface is a challenging task. Therefore, a machine learning-based protocol to minimize the computational cost associated with performing these simulations is desirable. In our study, we compare two versatile model architectures – Feed Forward Neural Networks (FFNN) and Long Short Term Memory (LSTM) networks in terms of their accuracy in forecasting the atomic charge density. These protocols accelerate the predictions of various properties from reactive force field simulations, thereby serving as a valuable tool for the development of reactive, charge-dependent, machine learned interatomic force fields for classical molecular dynamics. SNL is managed and operated by NTESS under DOE NNSA contract DE-NA0003525. SAND2023-11069A