Bulletin of the American Physical Society
APS March Meeting 2022
Volume 67, Number 3
Monday–Friday, March 14–18, 2022; Chicago
Session K09: Physics of Machine Learning IIRecordings Available

Hide Abstracts 
Sponsoring Units: GSNP GDS DCOMP DSOFT Chair: Yuhai Tu, IBM T. J. Watson Research Center Room: McCormick Place W180 
Tuesday, March 15, 2022 3:00PM  3:12PM 
K09.00001: Finding Spin Glass Ground States Through Deep Reinforcement Learning Mutian o Shen, Zohar Nussinov, YangYu Liu, Changjun Fan, Yizhou Sun, Zhong Liu Spin glasses are disordered magnets with random interactions that are, generally, in conflict with each other. Finding the ground states of spin glasses is not only essential for the understanding of the nature of disordered magnetic and other physical systems, but also useful to solve a broad array of hard combinatorial optimization problems across multiple disciplines. Despite decadeslong efforts, an algorithm with both high accuracy and high efficiency is still lacking. Here we introduce DIRAC  a deep reinforcement learning framework, which can be trained purely on smallscale spin glass instances and then applied to arbitrarily large ones. 
Tuesday, March 15, 2022 3:12PM  3:24PM 
K09.00002: The random energy landscape of softspin networks and its application to combinatorial optimizations Atsushi Yamamura, Hideo Mabuchi, Surya Ganguli Building neuromorphic analog hardware to solve largescale optimization problems has been a central challenge spanning physics and computer science. We analyze the computational performance of one such machine, a network of coupled optical parametric oscillators that act as softspins confined in a doublewell potential. Their remarkable capacity for solving Ising encodable combinatorial optimization problems has been intensively analyzed in many optical experiments and computer simulations. However, we lack theoretical insights into why such networks can be efficient optimizers. We tackle this question by elucidating the geometry of the network's high dimensional energy landscape through the KacRice formula with full replica symmetry and supersymmetry breaking. Our analysis reveals a hierarchy of geometric phase transitions in the landscape that facilitates optimization dynamics. In particular, we reveal a geometric phase where the nonconvex landscape's local minima are almost all global minima, which allows the network to find solutions significantly better than naive spectral methods. We also find the existence of a phase where we cannot adiabatically follow the global minima, which creates a finite energy gap between the networks' solution and the optimal lowest energy. 
Tuesday, March 15, 2022 3:24PM  3:36PM 
K09.00003: From classification to models: When do SVMs discover physical features? Arabind Swain, Ilya M Nemenman The complexity of glasses makes it challenging to explain their dynamics. Machine Learning (ML) in recent years has emerged as a promising tool to understand glassy dynamics, predicting rearrangements of particles in the glass with a high degree of accuracy. It also helped in discovering the link between structural features of the glass and the rearrangement dynamics. Support Vector Machine (SVM) was one of the first ML methods to discover such a nontrivial relationship. Specifically, an SVM model hyperparameter, the distance from the separating hyperplane, was found to be linearly related to the Arrhenius energy, which governed the rearrangement process. Here we investigate under which conditions SVM can discover such relevant physical quantities in glassy systems. We study a toy glass model with known energy where an SVM is trained to predict the system dynamics based on a large array of structural features. We demonstrate analytically that the distance from the inferred separating hyperplane becomes linearly related to Arrhenius energy under some conditions, which we investigate. This is a step towards understanding when ML can discover new physics. 
Tuesday, March 15, 2022 3:36PM  3:48PM 
K09.00004: Machine learning probing universality class of four models Lev Shchur, Evgeni Burovski, Vladislav Chertenkov We test details of a possible investigation of the universality using a deep learning approach. 
Tuesday, March 15, 2022 3:48PM  4:00PM 
K09.00005: Inverse design of nucleation seeds Ella M King, Chrisy Xiyu Du, Michael P Brenner Homogeneous nucleation is a rare, slow process. As a result, most commercial applications favor heterogeneous nucleation, using portions of a crystal to seed crystal growth. Nearly all of our theoretical understanding of nucleation seeds is limited to spherical and nearspherical seed structures. However, flexible seeds and seeds with greater surface area may better enable crystal growth by providing more pathways to crystallization. Using a combination of forward simulation and inverse design, we investigate the optimal nucleation seed structure to promote crystal growth. A deeper understanding of heterogeneous nucleation will have applications ranging from improved growth of silicon wafers for electronics to alternative methods of seeding clouds in the atmosphere. 
Tuesday, March 15, 2022 4:00PM  4:12PM 
K09.00006: Machine learning enabled largescale quantum kinetic Monte Carlo simulations of the FalicovKimball model Sheng Zhang, GiaWei Chern, Puhan Zhang We show that the celebrated FalicovKimball model exhibits rich and intriguing phaseordering dynamics. With the aid of modern machine learning methods, we demonstrate the firstever largescale kinetic Monte Carlo simulations on the FalicovKimball model. We uncover an unusual phaseseparation scenario where domain coarsening occurs simultaneously at two different scales: the growth of checkerboard clusters at smaller length scales and the expansion of superclusters, which are aggregates of the checkerboard patterns of the same sign, at a larger scale. We show that the emergence of superclusters is due to a hidden dynamical breaking of the sublattice symmetry. A selftrapping mechanism related to the superclusters gives rise to the arrested growth of the checkerboard patterns and of the superclusters themselves. Glassy behaviors similar to the one reported in this work could be generic for other correlated electron systems. 
Tuesday, March 15, 2022 4:12PM  4:24PM 
K09.00007: Biased Monte Carlo sampling in RBMs Beatriz Seoane, Aurélien Decelle, Cyril Furtlehner, Nicolas Bereux RBMs are generative models capable of fitting complex dataset's probability distributions. Thanks to their simple structure, they are particularly well suited for interpretability and pattern extraction, a feature particularly appealing for scientific use. Yet, in practice, it is hard to extract good equilibrium models for structured datasets (which are the standard case in most biologically interesting datasets) due to a divergence of the Monte Carlo mixing times. In this work, we show this barrier can be easily surmounted using biased Monte Carlo methods, just as commonly done in Statistical Mechanics, to reach equilibrium in the vicinity of first order phase transitions. 
Tuesday, March 15, 2022 4:24PM  4:36PM 
K09.00008: Nonequilibrium Monte Carlo for unfreezing variables near computational phase transitions Masoud Mohseni, Daniel K Eppens, Federico RicciTersenghi, Johan Strumpfer, Alan Ho, Raffaele Marino, Vasil Denchev, Sergei V Isakov, Sergio Boixo, Hartmut Neven

Tuesday, March 15, 2022 4:36PM  4:48PM 
K09.00009: Gauge freedoms, symmetries, and the interpretability of sequencefunction relationships Justin B Kinney, Anna Posfai, David M McCandlish Quantitative models of sequencefunction relationships are ubiquitous in postgenomic biology, e.g., for describing the activities of gene regulatory sequences or to model the fitness landscapes of proteins. These models usually exhibit many gauge freedoms—directions in parameter space along which transformations do not affect model output. But in contrast to the central role that gauge freedoms play in theoretical physics, the origins, properties, and consequences of gauge freedoms in sequencefunction relationships have received little attention. Here we study gauge freedoms in both linear and linearnonlinear sequencefunction relationships. In particular, we connect the gauge freedoms that arise in linear models with onehot sequence features to character permutation symmetry, and discuss the importance of such symmetry for biological interpretability. We also propose specific gauge fixing strategies that are especially useful in the context of these linear models. Finally, we identify two distinct classes of gauge freedoms that arise in linearnonlinear models which are not present in linear models. This work thus establishes mathematical and conceptual tools for better understanding how sequence encodes biological function. 
Tuesday, March 15, 2022 4:48PM  5:00PM 
K09.00010: Exploring relative advantages of dual vs single dimensionality reduction Eslam Abdelaleem, K. Michael Martini, Ilya M Nemenman The current experimental advancements in studying complex systems enable the recording of the activity of a large number of constituents of the system, while simultaneously recording the resultant system behavior. Usually, the modeling process begins with dimensionality reduction. Here we explore the dual dimensionality reduction, aiming to identify simultaneously which collective dynamics from the individual constituents are responsible for which reduced dimensional response features. Using linear modeling, we show that the dual dimensionality reduction approach requires significantly fewer data points than the regular methods (i.e., reducing each feature independently, and then identifying relations between the reduced descriptions). However, increasing the number of recorded dimensions is reflected as more noise in the measurements (quantified by the Signal to Noise Ratio  SNR). We numerically investigate the effect of SNR and the number of samples or observations N on single versus dual dimensionality reduction, showing in which regimes and conditions dual reduction is better and when both are the same. Then we outline how we study these results analytically. 
Tuesday, March 15, 2022 5:00PM  5:12PM 
K09.00011: An unsupervised neural network learns reproducible and interpretable representations of active matter systems ChihWei Joshua Liu, Nikta Fakhri, Junang Li, Michal Szurek Active matter systems are highdimensional. Dimensionality reduction is key to efficient characterization of systems' dynamics, but existing approaches for dimensionality reduction, such as principalcomponent analysis and singularvalue decomposition, can generate representations that are difficult to interpret. We previously presented an unsupervised neural network approach for representing highdimensional systems. Building on this work, here we present a variational Bayesian method for dimensionality reduction in dynamics data. We show that our unsupervised algorithm learns reproducible and interpretable representations of the system and observe putative currents in the representation phase space. Our datadriven approach facilitates experimental characterization of highdimensional dynamics, and may also enable irreversibility quantification in general active matter systems. 
Tuesday, March 15, 2022 5:12PM  5:24PM 
K09.00012: Criticality in Deep Neural Networks using Jacobian(s) Darshil H Doshi, Andrey Gromov, Tianyu He Deep Neural Networks (DNNs) have proven to be extremely successful in a variety of patternrecognition tasks. In contrast, the analytical understanding lags far behind. The simplifying limit of infinite width offers a way to analyze such networks. In this limit DNNs become Gaussian Processes; taking on a deterministic form. 
Tuesday, March 15, 2022 5:24PM  5:36PM 
K09.00013: Understanding Layer Normalization in Deep Neural Networks Tianyu He, Darshil H Doshi, Andrey Gromov Deep neural networks (DNNs) have been proven to be very powerful in classification, language modeling, and computer vision problems. However, training DNNs are computationally hard due to the huge number of parameters. A good initialization of a DNN can save enormous amounts of computational power. Layer normalization (LayerNorm) is introduced to make training faster and generalization easier. We show that the effect of LayerNorm can be quantitatively studied in infinite width limit, by plotting phase diagrams of DNNs. We then show empirically that for many cases, with LayerNorm layers, that a DNN initialized close to the phase boundary can have the best performance in both training and validation. 
Tuesday, March 15, 2022 5:36PM  5:48PM 
K09.00014: Information Bottleneck for Datadriven Renormalization without Locality K. Michael Martini, Joseph L Natale, Ilya M Nemenman The renormalization group (RG) has been immensely successful in defining variables relevant for the description of macroscopic systems. It achieves this by successively coarse graining and removing the smallest length scales in the problem to capture the long wavelength behavior. RG has been generally used for systems where the interactions are local and highly symmetric. Here we introduce a method for coarse graining experimental data, while extracting features relevant to their macroscopic behavior, without explicit references to locality or symmetry. We use the value of the mutual information between the microscopic components of the system as a proxy for locality. Our approach starts by selecting the most "local" microscopic components and uses the information bottleneck method to coarsegrain them while preserving as much information about the next "scale" as possible. We analytically determine the optimal tradeoff parameter between compression and information retention by finding the leading order error in our estimate of the information. By repeatedly applying this procedure we can recover the renormalization group flow for coupling strengths and correlations for data taken from the 2D Ising system. Applications to neural recordings are forthcoming. 
Follow Us 
Engage
Become an APS Member 
My APS
Renew Membership 
Information for 
About APSThe American Physical Society (APS) is a nonprofit membership organization working to advance the knowledge of physics. 
© 2024 American Physical Society
 All rights reserved  Terms of Use
 Contact Us
Headquarters
1 Physics Ellipse, College Park, MD 207403844
(301) 2093200
Editorial Office
1 Research Road, Ridge, NY 119612701
(631) 5914000
Office of Public Affairs
529 14th St NW, Suite 1050, Washington, D.C. 200452001
(202) 6628700