Can (Almost) Unsupervised Artificial Intelligence Learn Chemistry and Physics from Microscopic Observations?*

Machine learning has emerged as a powerful tool for the analysis of mesoscopic and atomically resolved
images and spectroscopy in electron and scanning probe microscopy. The applications ranging from
feature extraction to information compression and elucidation of relevant order parameters to inversion of
imaging data to reconstruct structural models have been demonstrated. In this presentation, I will discuss
several applications of autoencoders and variational autoencoders for the analysis of image and spectral
data in STEM and SPM. The special emphasis is made on the rotationally invariant variational
autoencoders that allow to disentangle rotational degrees of freedom from other latent variables in
imaging and spectral data. The analysis of the latent space of autoencoders further allows establishing
physically relevant transformation mechanisms. Ultimately, we demonstrate that given the postulated
existence of atoms, neural network can discover molecular fragments and reaction mechanisms.
Extension of encoder approach towards establishing structure-property relationships in the structure-
property data sets will be illustrated.