Session G02: Statistical Mechanics of the Brain

Recent progress in the statistical physics of real neural networks

Single neurons have fascinating properties, and make measurable contributions to perception and action. Nonetheless many of the most interesting phenomena in the brain seem to be collective or emergent. There is a long history of trying to use ideas from statistical physics to describe these phenomena, with many beautiful and influential results. Nonetheless it remained difficult to see how theory and experiment could be connected, quantitatively. This became more urgent as the 21st century brought new experimental tools for recording the electrical activity of more and more neurons simultaneously. Maximum entropy methods emerged as a strategy for building models that are grounded in statistical physics but connected directly to these data, so that we end up with (for example) a spin-glass like model for a particular population of neurons rather than some guess at the family from which this model might be drawn. After some initial uncertainties, it now is clear that these models can provide extraordinarily precise descriptions of ~100 cells, for example correctly predicting the ~100,000 triplet correlations in such networks within experimental errors. When models work this well it makes to take them seriously as statistical mechanics problems and ask, for example, where real networks are in phase diagram of possible networks. In cases where the simplest maximum entropy models fail, several alternatives have been suggested, and I will review some of these. We also need methods that have even better scaling with network size. In a different direction, recordings from 1000+ (or even 100,000+) neurons offer opportunities to explore coarse-graining, or more generally to ask if we can use the renormalization group to inspire new methods of data analysis. This has led to the observation of very precise and reproducible scaling behaviors, holding out that these networks might be described by theories that live at an RG fixed point.   

Multi-Relevance: Coexisting but Distinct Notions of Scale in Large Systems

An essential aspect of renormalization group (RG) methods is the cutoff scheme, which specifies how collective variables are integrated out. In most applications, such as in field theory, momentum is used as a cutoff scale, and RG produces effective low-momentum theories by averaging over high-momentum variables. Recently, RG methods have seen use in problems at the boundaries of statistical physics, biology, and computer science, where the models are complicated distributions over high-dimensional spaces. These models are frequently not analogous to traditional many-body systems, making it difficult to specify what precisely is meant by "scale". This makes RG hard to implement and interpret. Here, we present recent theoretical progress on both of these fronts. First, we show that non-perturbative RG is well-suited for models with finitely many degrees of freedom, and demonstrate a simple calculation. Next, we introduce a method of calculating the cutoff scheme to be used based on the structure of the model at hand. In doing so, we demonstrate that some models support multiple notions of scale, and term this property "multi-relevance". Finally, we examine how multi-relevance appears in problems relevant to fields that interface with statistical physics.   

Exploring Criticality in Markovian Brain Dynamics

Biological systems need to react to stimuli over a broad spectrum of timescales. If and how this ability can emerge without external fine-tuning is a puzzle. This problem has been considered in discrete Markovian systems where results from random matrix theory could be leveraged. Indeed, generic large transition matrices are governed by universal results, which predict the absence of long timescales unless fine-tuned. Our current model considered an ensemble of transition matrices and motivated a temperature-like uniformity variable that controls the dynamic range of matrix elements. Findings were applied to fMRI data from 820 human subjects scanned at wakeful rest. The data was quantitatively understood in terms of the random model, and brain activity was shown to lie close to a phase transition when engaged in unconstrained, task-free cognition – supporting the brain criticality hypothesis in this context. In this work, the model is advanced in order to discuss the effect of matrix asymmetry, which is important for certain measures in stochastic thermodynamics (ex: detailed balance). We introduce a new parameter that controls the asymmetry of these discrete Markovian systems and show that when varied over an appropriate scale this factor is able to collapse Shannon entropy measures. This collapse indicates that structure emerges over a dynamic range of both uniformity and asymmetry. Moreover, results are used to identify phase transitions in monkeys transitioning between anesthetic and wakeful states.

    

Sparse, consistent correlation structure in the retinal population code across natural movies

Neural populations are known to adapt their coding scheme in response to different scenes, but exactly how remains a mystery, especially at the population rather than single cell level. We analyze data from the larval salamander retina responding to five different natural movies, and use maximum entropy models to characterize the population in terms of activations and couplings between neurons. We find evidence that while individual cells are adapting their response to the stimulus, the couplings, i.e. the population structure, is consistent across natural movies. In particular, we find a sparse, strongly connected backbone of couplings, which constrains the vocabulary of the neural population. We also show that this consistent coupling could provide a stable structure to allow for consistent decoding despite adaptation. Finally, we show that we can make use of this consistent structure to build models of large groups of neurons in a new, scalable way by taking advantage of the consistency of the learned couplings across small groups.   

Coarse-graining facilitates generalization in populations of retinal ganglion cells

The output of the retina contains all of the information the brain encodes about the visual world. The joint probability distribution of these outputs, retinal ganglion cells (RGCs), is known to change with the statistics of the scene driving retinal activity. This creates a significant challenge in creating generative models of population activity that generalize to new types of stimuli. We build on an information-theoretic coarse-graining proposed by Ramirez and Bialek to take the population of neurons from an exponential number of states to a linear number of states. Using data from RGCs in the larval salamander retina in response to a variety of natural moving scenes, we test how well coarse-grained representations generalize. We find that trial-averaging significantly improves generalization to other natural movies. These coarse-grainings can be input to a Generalized Linear Model (GLM), an interpretable, generative model of retinal activity. With this input, the GLM performs well at recapitulating retinal response and generalizes well to novel stimulus statistics, including from spatial white noise checkerboards to natural movies. This approach may be helpful in modeling other areas of the brain and has implications both for basic neuroscience and machine learning.   

Brain network dynamics for navigational learning and memory

Effective spatial navigation is dependent on several cognitive processes including learning, attention, and memory, but how do people learn and remember new environments? Previous studies explored brain activity in already known environments, however, comparatively little is known about the acquisition of this knowledge. We analyzed fMRI of humans completing a challenging maze learning task. Participants were given 16 minutes to explore a virtual hedge maze and learn the locations of 9 objects. Next, their object location memory was tested in 48 ( <45 s) trials, each starting at one object with instructions to find another object (obscured for testing) using paths of the maze (e.g. clock to lamp). Accuracy ranged from near 0% to 100%, enabling us to quantify brain network and behavioral differences that distinguish between poor, average, and exceptional performers. We used dynamic community detection to identify brain network changes. Preliminary results suggest that the best navigators exhibit high flexibility throughout the brain, whereas average and poor navigators exhibit low flexibility only in the salience/ventral attention network. In addition, we examined behavioral exploration patterns in the learning phase to determine whether they correlate with navigation accuracy in the test phase, finding that better navigators tend to explore more evenly than poor navigators. Together, the brain and behavioral dynamics in this study provide rich insight into navigational learning and memory.

    

Heavy-tailed neuronal connectivity arises from Hebbian self-organization

In networks of neurons, the connections are heavy-tailed, with a small number of neurons connected much more strongly than the vast majority of pairs. Yet it remains unclear whether, and how, such heavy-tailed connectivity emerges from simple underlying mechanisms. Here we propose a minimal model of synaptic self-organization: connections are pruned at random, and the synaptic strength rearranges under a mixture of Hebbian and random dynamics. Under these generic rules, networks evolve to produce scale-free distributions of connectivity strength, with a power-law exponent γ = 1 + 1/p that depends only on the probability p of Hebbian (rather than random) growth. Extending our model to include correlations in neuronal activity, we find that clustering (another ubiquitous feature of neuronal networks) also emerges naturally. We confirm these predictions in the connectomes of several animals, suggesting that heavy-tailed and clustered connectivity may arise from general principles of self-organization, rather than the particulars of individual species or systems.   

Signatures of criticality in the physical structure of the brain

The highly complex structure of the brain is integral to its function, yet many aspects of the physical structure of the brain and how they aid its function remain poorly understood. Recent experimental and computational advances have enabled the three dimensional reconstruction of millimeter-scale brain volumes of multiple organisms at the cellular level [1-3], thereby allowing for detailed investigation of the structure of brain tissue.

Here we utilize these datasets to examine physical properties of the structure of the brain using techniques from statistical physics. We demonstrate that the physical structure of the brain displays features associated with structural criticality in physical systems, including long-range correlations and fractal properties. We show that the associated scaling exponents are consistent across the brains investigated, indicating potential structural universality in the brain.

[1] C. Shan Xu, et al., bioRxiv 2020.01.21.911859 (2020)

[2] MICrONs Consortium, et al., bioRxiv 2021.07.28.454025 (2021)

[3] A. Shapson-Coe, et al., bioRxiv 2021.05.29.446289 (2021)   

Early Path Dominance as a Principle for Neurodevelopment

To understand how the neural networks of brains acquire their topological structures upon development, we perform a computational study of human diffusion MRI across adults (UK Biobank, N =19,380, adolescents (Adolescent Brain Cognitive Development Study, N =15,593) and neonates (Developing Human Connectome Project, N =758), as well as mouse viral tracing (Allen Institute). We perform targeted attack, a systematic unlinking of the network, to analyze its effects on global communication across the network through its giant cluster.  We find that brain networks differ from scale-free and small-world structures. Time-reversing the attack computation suggests a mechanism for how brains develop, the validity for which we establish experimentally for targeted attack on increasing white matter tract lengths and densities shown to be invariant to aging and disease.  We derive an analytical equation using percolation theory for the fraction of brain regions in the giant cluster as a function of connectivity. Based on a close match between theory and experiment, our results demonstrate that tracts are limited to emanate from regions already in the giant cluster and that tracts that appear earliest in neurodevelopment are those that become the longest and densest.   

Multistable irregular activity in large winner-take-all networks

Identifying multistable network models producing cortical-like irregular spiking is an unsolved challenge in neuroscience. Multistability, thought to support working memory, is usually modeled via persistent states in which certain neurons receive elevated mean inputs. In contrast, irregular spiking is usually modeled in large networks in which neurons receive inputs that strongly fluctuate around a vanishingly small mean. This suggests stable states might instead be distinguished by higher-order input statistics, but networks operating in such a regime are not well understood. Here we show that a network of winner-take-all-like neuron groups can create a landscape of steady states in which sparse, irregular spiking stabilizes memories of past external inputs. Via simulation and theory we show this is not a finite-size effect but results from a symmetry breaking in networks driven by multi-dimensional input fluctuations. Our results survive randomization of the competitive interactions, suggesting fine-tuned reciprocal connections are not needed. Thus, explicit competition among neurons, potentially via fast lateral inhibition, can enable irregular spiking to protect rather than degrade information, illustrating a novel collective mechanism that could support working memory.   

A random matrix theory in eigenspace enables direct control of collective network activity

A common approach to analytically treat neural network models is to assume a random connectivity matrix. But how does our choice of randomness affect the network's behavior? Rather than prescribing the distribution of synaptic strengths, we specify connectivity in the space that directly controls the dynamics – the space of eigenmodes. We develop a thermodynamic theory for a novel ensemble of random matrices, whose eigenvalue distribution can be chosen arbitrarily. As this distribution is varied, we show analytically how the behavior of a stochastic linear rate network changes. We discover a critical point whose nature is shaped by the distribution of oscillation frequencies of near-critical modes: correlation and response functions can be tuned from an exponential to a power-law decay in time. Their decay exponents (d-1 and d, respectively) slow down ∝ d, which controls the density of the real part of near-critical eigenvalues p( x ) ∼ x^d-1, for x → 0. Also, the network shows a transition from high to low dimensional activity when d < 2, with a minimum at d = 1. We argue that d can be interpreted as the network's spatial dimension in the sense of critical phenomena and the renormalization group. In particular, below a critical dimension d = 1, correlations diverge, hinting at a potential transition to nongaussian criticality in a nonlinear system. Our novel approach uncovers a diverse range of behaviors that only emerges when the collective effect of the subleading-order synaptic strengths' statistics is not neglected.   

Learning Dynamic Graphs, Too Slow

The structure of knowledge is commonly described as a network of key concepts and semantic relations between them. A learner of a particular domain can discover this network by navigating the nodes and edges presented by instructional material, such as a textbook, workbook, or other text. While over a long temporal period such exploration processes are certain to discover the whole connected network, little is known about how the learning is affected by the dual pressures of finite study time and human mental errors. Here we model the learning of linear algebra textbooks with finite length random walks over the corresponding semantic networks. Through a combination of stochastic simulations and statistical mechanics theory, we show that if a learner does not keep up with the pace of material presentation, the learning can be an order of magnitude worse than it is in the asymptotic limit. Further, we find that this loss is compounded by three types of mental errors: forgetting, shuffling, and reinforcement. Broadly, our study informs the design of teaching materials from both structural and temporal perspectives.   

Thermal management in neuromorphic materials, devices, and networks

 

Computing technology has experienced unprecedented growth in recent years, often referred to as an "artificial intelligence revolution." The human brain inspires the fundamental approach for an energy-efficient new computing paradigm: using neural networks to classify large amounts of data into sorting categories. ^[1,2] Although the implementation of machine learning schemes significantly improves the  computing machine in multiple applications, these approaches are still inefficient in energy consumption, and thermal management. This motivates the search for new hardware-based systems that imitate the brain. ^[1,2,3] In this talk, we describe the current state of thermal management of neuromorphic systems; the role of the brain criticality and resistive switching-based neuromorphic devices ^{[4, 5]}; the energy and electrical considerations for spiking-based computation and their relation with fundamental features of the brain's thermo regulation ^[6]; graph neural networks models in neuromorphic computing ^[7]; and thermoelectric control of materials and neuromorphic devices. ^[3]