Non-contact molecular manipulation based on Reinforcement Learning*

At the world’s first race of nanocars at the CEMES-CNRS, in France, participants had to direct a nanocar across a specific “racetrack” [1]. In order to control their nanocar, they have to pull it via an STM-tip, without being in direct contact with the nanocar.

The physics that govern the molecule’s movement and rotation is complex and involves the interaction between the molecule and the tip as well as the molecule and the substrate [2]. Therefore, it requires time and expertise for humans to manoeuvre a molecule with reasonable success. However, predicting the outcome of a performed action is often unintuitive and hard to predict for humans.

Therefore, we develop an artificial intelligence (AI) based on reinforcement learning (RL) and show how it can be implemented to manipulate single molecules. The AI utilizes an off-policy RL algorithm known as Q-Learning. This algorithm provides insights into the behaviour of the molecule in the vicinity of an electric field but without the necessity of a physical model.

Our results can be the basis for more sophisticated techniques of molecular manipulations. This allows to identify and dislocate single molecules at will, building the basis for future bottom-up constructions of nanotechnology.