Optimal machine intelligence near the edge of chaos*

We develop a general theory that reveals the exact edge of chaos for generic non-linear systems is the boundary between the chaotic phase and the (pseudo)periodic phase arising from Neimark-Sacker bifurcation. This edge is analytically determined by the asymptotic Jacobian norm values of the non-linear operator and influenced by the dimensionality of the system. The optimality at the edge of chaos is associated with the highest information transfer between input and output at this point, inferred by the maximal information content in the system’s asymptotic periodic states similar to that of the logistic map. As empirical validations, our experiments on the deep learning models in computer vision trained on benchmark data set demonstrate the optimality of the models near the edge of chaos. Our finding contributes to the theoretical and empirical foundation of the edge of chaos hypothesis, while our theory provides fundamental understanding of machine intelligence in deep learning.