Session B22: Building the Bridge to Exascale: Applications and Opportunities for Materials, Chemistry, and Biology II

Towards catalysis modeling with QMC

Predictive modeling of catalytic processes on surfaces is challenging because of large uncertainties in computed energetics using density functional theory (DFT), and exponential dependence of catalyst performance on energies. In particular, the use of various DFT functionals and approximations results in qualitatively different results. The problem is compounded when treating transition metal oxides where a host of DFT errors persist. In this work, made possible by leadership scale high performance computers, we use quantum Monte Carlo (QMC) to determine the adsorption energies of the CO molecule on Cu$_2$O (110) surface for various geometries determined by different DFT approximations. The relationships between geometry and energies, and between DFT and QMC results, will be discussed.   

Accelerating quantum Monte Carlo simulations on many-core processors via OpenMP nested threading

The massively parallel nature of quantum Monte Carlo is ideally suited to petascale computers which have enabled a wide range of applications to relatively large molecular and extended systems. The current scheme to achieve the shortest time-to-solution is placing one walker per core on multi/many-core processors. However, this strategy meets a great challenge from upcoming Exascale computers where much larger problem will be solved. The time to advance a walker and the memory need of it scale cubically and quadratically as a function of the electron counts in the system. On current computers, large problems already force us to leave cores idle for fitting them in memory. In addition, adding more walkers reduces the production steps for a given amount of samples but do not reduce the equilibration steps, which causes even more waste. In order to reduce the time-to-solution and reduce the memory footprint, we introduce nested threading to distribute the computation of each walker over several cores. We explore threading algorithms with minimal overhead and demonstrate a good scaling.   

Structure and magnetism in bulk and uniaxially strained LaCoO3-x through ab-initio diffusion quantum Monte Carlo

Advances in high-performance computing (HPC) is allowing the use of ambitious methods in materials science with unprecedented accuracy. Density Functional Theory (DFT) accuracy is not sufficient for challenging problems, as it depends critically on the empirical corrections applied, such as Hubbard-U and exchange mixing. For example, DFT has not yet been able to discern the microscopic origin of the ferromagnetic (FM) state in uniaxially strained LaCoO₃ thin films. In contrast, Diffusion quantum Monte Carlo (DMC) method treats electrons explicitly, solving the many-body Schrodinger equation with minimum approximations. DMC has been applied to an increasingly larger set of materials with excellent agreement. In this presentation, we report ground state energies, magnetism, defect formation energies in uniaxially strained LaCoO_3-x using DMC. DMC yields an antiferromagnetic ground state for bulk uniaxially strained LaCoO₃which agrees with the recent experiments. A transition between high-spin AFM and FM structures is also identified in the uniaxially strained structures, which may help explain the spin transition in thin films.   

Accelerating Large-Scale GW Calculations on Many-Core and Hybrid CPU+GPU HPC Systems

The novel electronic and optical properties found in complex materials represent the basis for the development of many emerging technologies. The rational design of such technologies requires an accurate quantum mechanical predictive capability (e.g., the GW approximation within many-body perturbation theory and beyond) that can run in reasonable timescales on available high performance computing (HPC) facilities. In this talk we summarize the advances in method development and code optimization of the BerkeleyGW software package targeted for many-core and hybrid CPU+GPU architectures. In particular showing how we have combined methods to reduce the prefactor of the calculations with an optimal implementation suitable for many-core architectures which allowed us to achieve excellent parallel scalability (>500k cores), high fraction of peak performance and thus excellent time to solution for systems up to thousands of atoms. We show how these developments can be combined to take advantages of GPU accelerators.
  

Large scale GW and BSE calculations using interoperable software building blocks

The modeling of light-matter interaction in complex heterogenous systems is key to several materials design problems. A microscopic modeling of two-particle correlation functions requires the solution of the Bethe-Salpeter equation (BSE) based on many body perturbation theory. However, the application of BSE-GW calculations to complex nanostructured, disordered or defective materials has been hindered by high computational costs. We present a method, implemented into interoperable software building blocks, to compute optical spectra and exciton binding energies based on the solution of the Liouville equation and the calculation of the screened Coulomb interaction in finite field. The method does not require the explicit evaluation of dielectric matrices nor of virtual electronic states, and can be easily applied to large systems, beyond the random phase approximation. Localized orbitals obtained from Bloch states using bisection techniques are used to reduce the complexity of the calculation and enable the efficient use of hybrid functionals. We will discuss the advantages of this paradigm, and provide results for the calculation of spectroscopic features of promising materials for spin qubits.   

Pushing the Accuracy Limits of Scalable Fixed-Node Diffusion Monte Carlo for Noncovalent Interactions

Single-determinant (SD) fixed-node diffusion Monte Carlo (FNDMC) gains popularity as a benchmark tool scalable to large noncovalent systems although its accuracy limits are not yet fully mapped out. Some of the recent results suggest that accuracy of SD FNDMC is capable of achieving benchmark accuracy for a rich class of noncovalent systems, larger than expected before. The talk will summarize recent progress and current accuracy limits of bias-cancellation-based SD FNDMC for molecules and noncovalent crystals. The best available "recipe" suitable for a wide class of noncovalent systems will be sketched out.   

Deep Machine Learning for Atomically-Resolved Imaging Experiments: Physics Extraction and Feedback

Development of the imaging tools such as electron, optical, and scanning probe microscopy in the last decade of 20th century has opened floodgates of imaging data, in the form of images, movies, and hyperspectral data sets. These contain information on the minute details of atomic structure and electronic, magnetic, and phonon functionalities, chemical transformation mechanisms, and quantum phenomena. However, the bottleneck in analysis and knowledge extraction from large volumes of raw imaging data is often human domain expert. Here I will show how utilization of deep artificial neural networks (aka deep learning) offers a path to overcome limitations of human analysis. I will specifically discuss applications of deep learning for rapid fully automated identification of individual atomic types and positions from scanning transmission electron microscopy images, using theoretical or labeled experimental images as a training set. We used this approach to construct reaction pathways for point defects in 2D materials, trace the structural evolution of atomic species during the electron beam manipulation, and create the library of defect configurations in silicon- and vacancy doped graphene. I will discuss specific examples where we showed that coupling of sulphur vacancy to molybdenum dopants in tungsten disulfide and reactions of silicon impurity on the edge and in the bulk of graphene can be explored quantitatively and mapped on the Markov model, giving rise to the transition probabilities on single atomic defect level. The work on genetic engineering optimization framework for automatic design of deep learning network architecture and hyperparameters optimal specifically for electron microscopy datasets will be addressed. Finally, I will discuss how a synergy of deep learning image analytics and real-time feedback allows harnessing beam-induced atomic and bond dynamics to enable direct atom-by-atom fabrication.
  

Decoding Inverse Imaging Problems in Materials with Distributed Deep Learning

Materials physics abounds with challenging inverse problems, from inverse materials design to reconstruction of material properties from imaging/scattering data. One of the oldest inverse problems in materials is the reconstruction of the local atomic scattering potential from convergent beam electron diffraction (CBED). In this talk, we will present results on a potential reconstruction approach based on deep neural networks (DNN). In particular, we used data parallelism to distribute the training of a DNN over 12,000 GPUs on the Summit supercomputer. By efficient utilization of Summit's burst buffer and half-precision arithmetics, the data processing rates of the DNN reached upwards of 100 GB/s, allowing for the processing of local CBED patterns from 60,000+ crystal structures in the matter of minutes. We will present challenges encountered in distributed training at these scales, such as I/O bottlenecks and DNN convergence and how they can be mitigated. Finally, initial results on our DNN-based reconstruction of local atomic potentials of strontium irridate superlattices will be presented.   

Convolutional neural network based super resolution for brain cell mapping

Synchrotron based full-field nano-CT (computed tomography) holds the key to map the cellular composition of the entire brain cells. In particular, our in-house Transmission X-ray Microscope (TXM) at the Advanced Photon Source (sector 32-ID) in the Argonne National Laboratory is designed to achieve a spatial resolution of 20 nm. But our ability to resolve features of nervous system in few nanometers - required to properly discern the connectivity in the whole brain - is constrained by the laws of optics of the X-ray microscope (namely depth of focus). Accordingly, in this contribution, we propose the use of deep convolutional neural network (CNN) based super-resolution to achieve a resolution beyond the limit of the TXM. The mapping from low resolution to high resolution will be deduced accounting for degradations such as blur kernel and noise level to faithfully model the artifacts observed in TXM based CT results. Finally, the L2 based loss function will be combined with regularization to preserve the edges of minute structure seen in brain cells.
  

Towards Exascale Bio-molecular Simulations with Artificial Intelligence Workflows

Molecular simulations take significant supercomputing resources, roughly between 40 and 60% of time. While emerging Exascale compute architectures promise to provide an unprecedented capability to simulate complex physical phenomena, they also pose a number of computational challenges for effective scaling such simulations. We describe our research in scaling bio-molecular simulations by tightly integrating artificial intelligence (AI) approaches using our Molecules library. We demonstrate Molecules can extract biophysically meaningful reaction coordinates from long time-scale simulations (post-processing and/or in situ) that can be interpreted with respect to experimental data. We then show that the AI-derived reaction coordinates can be used to accelerate molecular simulations, especially in discovering novel conformational states that can be used to drive additional exploration through high dimensional landscapes. Using reinforcement learning techniques we demonstrate that these states can sample novel folding pathways that have not been explored previously. Together, we show that our integrated workflow can effectively accelerate sampling of high dimensional conformational landscapes while simultaneously making use of emerging hardware features of Exascale architectures.   

Dynamical Implications of Hyperparameters in Reinforcement Learning

Reinforcement learning has been gaining popularity within the physics research community over past several years. Many of the algorithms require selection of hyperparameters, and it is unclear how these choices affect the learned policy. Often in practice, researchers choose hyperparameters for their simulations using brute force methods such as exhaustive grid search or heuristics without fully understanding how they may alter the effective reward structure and value function landscape. Here, we investigate the possibility of using a nonlinear dynamics approach to guide our selection and adaptive optimization of appropriate hyperparameters that are better aligned to achieve particular aims of the agent. We then extend our investigation to the setting of multi-agent reinforcement learning and game theory to see how the choice of agents hyperparameters affect the interplay between each other in competitive, cooperative, and mixed settings.   

ExaTN - A Scalable Exascale Math Library for Hierarchical Tensor Network Representations and Simulations

A problem that will benefit from exascale computing is predictive simulation of two- and three-dimensional quantum many-body Hamiltonians. Enabling efficient numerical simulations requires new insights into wavefunction compression that scales on extremely heterogeneous HPC architectures. To address this need, we develop ExaTN: a math library of parallel numerical primitives for processing operations based on hierarchical tensor representations. Our library provides a scalable infrastructure for building a performance portable framework for simulating strongly correlated quantum systems on heterogeneous HPC platforms such as Titan and Summit. Our framework is the first to deliver a massively parallel implementation of the multiscale entanglement renormalization ansatz (MERA), a promising hierarchical tensor decomposition scheme capable of expressing local expectation values in strongly entangled quantum systems efficiently. In this talk, we will present our integrated framework for processing hierarchical tensor representations on exascale HPC systems via a multi-level asynchronous task-based programming model. This provides scientists with a capability for simulating strongly entangled systems in condensed matter physics.   

UV/vis absorption spectra database auto-generated for optical applications via the Argonne data science program

A large corpus of material and experimental data exists in historic scientific literature. Natural language processing and data-mining approaches can be applied to curated scientific literature to extract chemical information for targeted functionality and applications. This talk focuses on development of a UV/vis absorption spectra database by means of a complex quantum chemistry workflow built on the top of the ChemDataExtractor tool [1] by leveraging DOE Argonne leadership computing facilities as a part of the data science project allocation. We show results of retrieving chemical information and experimental properties from a large sample of scientific literature (~ 400,000) with chemdataextractor. Some electronic structure properties of a large subset of compounds are modeled using quantum chemistry workflows for benchmarking and validation. Finally, the quality of the database is discussed based on validation metrics and its applicability to optical applications.