Theory of gating in recurrent neural networks*

Understanding the emergent dynamics of networks of neurons is a central challenge in theoretical neuroscience. Most of the work in understanding the dynamics of these networks has focused on models with `additive interactions', where the input to a neuron is a weighted sum of the output of the rest of the network. However, there is ample evidence from neurophysiology that neurons can have gating or multiplicative interactions, where e.g. one neuron can dynamically decide whether another neuron is influenced by the rest of the network. Such gating interactions lead to qualitatively different behavior of single neurons, and are likely to have even more dramatic effects on the collective behavior of a network. Furthermore, researchers in machine learning have found that gating interactions facilitate training of model neural networks. Thus, gating can have significant implications for information processing. We leverage tools from the field theory of disordered systems to develop a theory of gating in a canonical neural network model. Our theory allows us to elucidate the dynamical aspects of gating which are important for the network's information processing capabilities.