Exploring Electron Beam Induced Atomic Assembly via Reinforcement Learning in a Molecular Dynamics Environment

Atom-by-atom assembly of functional materials and devices is perceived as one of the ultimate targets of nanoscience and nanotechnology. While traditionally implemented via scanning probe microscopy techniques, recently it has been shown that the beam of a scanning transmission electron microscope can be used for targeted manipulation of individual atoms. However, the process is highly dynamic in nature and proceeds via a large number of weakly-understood individual steps. Hence, harnessing an electron beam towards atomic assembly requires automated methods to control the parameters and positioning of the beam in such a way as to fabricate atomic-scale structures reliably. Here, we create a molecular dynamics environment wherein individual atom velocities can be modified, effectively simulating a beam-induced interaction, and apply reinforcement learning to model construction of specific atomic units consisting of Si dopant atoms on a graphene lattice. We find that it is possible to engineer the reward function of the agent in such a way as to encourage formation of local clusters of dopants, whilst at the same time minimizing the amplitude of momentum changes. Inspection of the learned policies indicates that of fundamental importance is the component of velocity perpendicular to the material plane, and further, that the high stochasticity of the environment leads to conservative policies. This study shows the potential for reinforcement learning agents trained in simulated environments for potential use as atomic scale fabricators, and further, that the dynamics learned by agents encode specific elements of important physics that can be learned.