Describing condensed matter from atomically resolved imaging data: from structure to generative and causal models

The development of high-resolution imaging methods such as electron and scanning probe microscopy and atomic probe tomography have provided a wealth of information on structure and functionalities of solids. The availability of this data in turn necessitates development of approaches to derive quantitative physical information, much like the development of scattering methods in the early XX century which have given one of the most powerful tools in condensed matter physics arsenal. Here, we argue that this transition requires adapting classical macroscopic definitions, that can in turn enable fundamentally new opportunities in understanding physics and chemistry. For example, many macroscopic definitions such as symmetry can be introduced locally only in a Bayesian sense, balancing the prior knowledge of materials physics and experimental data to yield posterior probability distributions. At the same time, a wealth of local data allows fundamentally new approaches for the description of solids based on construction of statistical and physical generative models, akin to Ginzburg-Landau thermodynamic models. Finally, we note that availability of observational data opens pathways towards exploring causal mechanisms underpinning solid structure and functionality.

Physics makes the difference: Bayesian optimization and active learning via augmented Gaussian process

Both experimental and computational methods for the exploration of structure, functionality, and properties of materials often necessitate the search across broad parameter spaces to discover optimal experimental conditions and regions of interest in the image space or parameter space of computational models. The direct grid search of the parameter space tends to be extremely time-consuming, leading to the development of strategies balancing exploration of unknown parameter spaces and exploitation towards required performance metrics. However, classical Bayesian optimization strategies based on the Gaussian process (GP) do not readily allow for the incorporation of the known physical behaviors or past knowledge. Here we explore a hybrid optimization/exploration algorithm created by augmenting the standard GP with a structured probabilistic model of the expected systems behavior. This approach balances the flexibility of the non-parametric GP approach with a rigid structure of physical knowledge encoded into the parametric model. The fully Bayesian treatment of the latter allows additional control over the optimization via the selection of priors for the model parameters. The method is demonstrated for a noisy version of the classical objective function used to evaluate optimization algorithms and further extended to physical lattice models. This methodology is expected to be universally suitable for injecting prior knowledge in the form of physical models and past data in the Bayesian optimization framework

Physics discovery in nanoplasmonic systems via autonomous experiments in Scanning Transmission Electron Microscopy

Physics-driven discovery in an autonomous experiment has emerged as a dream application of machine learning in physical sciences. Here we develop and experimentally implement deep kernel learning workflow combining the correlative prediction of the target functional response and its uncertainty from the structure, and physics-based selection of acquisition function guiding the navigation of the image space. Compared to classical Bayesian optimization methods, this approach allows to capture the complex spatial features present in the images of realistic materials, and dynamically learn structure-property relationships towards physical discovery. Here, this approach is illustrated for nanoplasmonic studies of the nanoparticles and experimentally implemented for bulk- and edge plasmon discovery in MnPS3, a lesser-known beam-sensitive layered 2D material. This approach is universal and is expected to be applicable to probe-based microscopic techniques including other STEM modalities and Scanning Probe Microscopies.

High-Throughput Study of Antisolvents on the Stability of Multicomponent Metal Halide Perovskites through Robotics-Based Synthesis and Machine Learning Approaches

Antisolvent crystallization methods are frequently used to fabricate high-quality perovskite thin films, to produce sizable single crystals, and to synthesize nanoparticles at room temperature. However, a systematic exploration of the effect of specific antisolvents on the intrinsic stability of multicomponent metal halide perovskites has yet to be demonstrated. Here, we develop a high-throughput experimental workflow that incorporates chemical robotic synthesis, automated characterization, and machine learning techniques to explore how the choice of antisolvent affects the intrinsic stability of binary perovskite systems in ambient conditions over time. Different combinations of the endmembers, MAPbI3, MAPbBr3, FAPbI3, FAPbBr3, CsPbI3, and CsPbBr3, are used to synthesize 15 combinatorial libraries, each with 96 unique combinations. In total, roughly 1100 different compositions are synthesized. Each library is fabricated twice using two different antisolvents: toluene and chloroform. Once synthesized, photoluminescence spectroscopy is automatically performed every 5 minutes for approximately 6 hours. Non-negative Matrix Factorization (NMF) is then utilized to map the time- and compositional-dependent optoelectronic properties. Through the utilization of this workflow for each library, we demonstrate that the selection of antisolvent is critical to the stability of metal halide perovskites in ambient conditions. We explore possible dynamical processes, such as halide segregation, responsible for either the stability or eventual degradation as caused by the choice of antisolvent. Overall, this high-throughput study demonstrates the vital role that antisolvents play in the synthesis of high-quality multicomponent metal halide perovskite systems.

Stress-Induced Phase Transitions in Nanoscale CuInP$_2$S$_6$

Using Landau-Ginsburg-Devonshire approach and available experimental results we reconstruct the thermodynamic potential of the layered ferroelectric CuInP$_2$S$_6$ (CIPS), which is expected to be applicable a wide range of temperatures and applied pressures. The analysis of temperature dependences of the dielectric permittivity and lattice constants for different applied pressures unexpectedly reveals the critically important role of the nonlinear electrostriction in this material. With the nonlinear electrostriction included we calculated temperature and pressure phase diagrams and spontaneous polarization of bulk CIPS. Using the coefficients of the reconstructed four-well thermodynamic potential, we study the strain-induced phase transitions in thin epitaxial CIPS films, as well as the stress-induced phase transitions in CIPS nanoparticles, which shape varies from prolate needles to oblate disks. We reveal the strong influence of the mismatch strain, elastic stress and shape anisotropy on the polar properties and phase diagrams of nanoscale CIPS. Also, we derived analytical expressions, which allow the elastic control of the nanoscale CIPS polar properties. Hence obtained results can be of particular interest for the strain-engineering of nanoscale layered nanoferroelectrics.

Automated and Autonomous Experiment in Electron and Scanning Probe Microscopy

Machine learning and artificial intelligence (ML/AI) are rapidly becoming an indispensable part of physics research, with domain applications ranging from theory and materials prediction to high-throughput data analysis. In parallel, the recent successes in applying ML/AI methods for autonomous systems from robotics through self-driving cars to organic and inorganic synthesis are generating enthusiasm for the potential of these techniques to enable automated and autonomous experiment (AE) in imaging. Here, we aim to analyze the major pathways towards AE in imaging methods with sequential image formation mechanisms, focusing on scanning probe microscopy (SPM) and (scanning) transmission electron microscopy ((S)TEM). We argue that automated experiments should necessarily be discussed in a broader context of the general domain knowledge that both informs the experiment and is increased as the result of the experiment. As such, this analysis should explore the human and ML/AI roles prior to and during the experiment, and consider the latencies, biases, and knowledge priors of the decision-making process. Similarly, such discussion should include the limitations of the existing imaging systems, including intrinsic latencies, non-idealities and drifts comprising both correctable and stochastic components. We further pose that the role of the AE in microscopy is not the exclusion of human operators (as is the case for autonomous driving), but rather automation of routine operations such as microscope tuning, etc., prior to the experiment, and conversion of low latency decision making processes on the time scale spanning from image acquisition to human-level high-order experiment planning.

Unsupervised learning of ferroic variants from atomically resolved STEM images

An approach for the analysis of atomically resolved scanning transmission electron microscopy data with multiple ferroic variants in the presence of imaging non-idealities and chemical variabilities based on a rotationally invariant variational autoencoder (rVAE) is presented. We show that an optimal local descriptor for the analysis is a sub-image centered at specific atomic units, since materials and microscope distortions preclude the use of an ideal lattice as a reference point. The applicability of unsupervised clustering and dimensionality reduction methods is explored and are shown to produce clusters dominated by chemical and microscope effects, with a large number of classes required to establish the presence of rotational variants. Comparatively, the rVAE allows extraction of the angle corresponding to the orientation of ferroic variants explicitly, enabling straightforward identification of the ferroic variants as regions with constant or smoothly changing latent variables and sharp orientational changes. This approach allows further exploration of the chemical variability by separating the rotational degrees of freedom via rVAE and searching for remaining variability in the system. The code used in the manuscript is available at https://github.com/saimani5/ferroelectric_domains_rVAE.

Deep Bayesian Local Crystallography

The advent of high-resolution electron and scanning probe microscopy imaging has opened the floodgates for acquiring atomically resolved images of bulk materials, 2D materials, and surfaces. This plethora of data contains an immense volume of information on materials structures, structural distortions, and physical functionalities. Harnessing this knowledge regarding local physical phenomena necessitates the development of the mathematical frameworks for extraction of relevant information. However, the analysis of atomically resolved images is often based on the adaptation of concepts from macroscopic physics, notably translational and point group symmetries and symmetry lowering phenomena. Here, we explore the bottom-up definition of structural units and symmetry in atomically resolved data using a Bayesian framework. We demonstrate the need for a Bayesian definition of symmetry using a simple toy model and demonstrate how this definition can be extended to the experimental data using deep learning networks in a Bayesian setting, namely rotationally invariant variational autoencoders.

Unsupervised Machine Learning Discovery of Chemical and Physical Transformation Pathways from Imaging Data

We show that unsupervised machine learning can be used to learn physical and chemical transformation pathways from the observational microscopic data, as demonstrated for atomically resolved images in Scanning Transmission Electron Microscopy (STEM) and ferroelectric domain structures in Piezoresponse Force Microscopy (PFM). To enable this analysis in STEM, we assumed the existence of atoms, a discreteness of atomic classes, and the presence of an explicit relationship between the observed STEM contrast and the presence of atomic units. In PFM, we assumed the uniquely-defined relationship between the measured signal and polarization distribution. With only these postulates, we developed a machine learning method leveraging a rotationally-invariant variational autoencoder (rVAE) that can identify the existing structural units observed within a material. The approach encodes the information contained in image sequences using a small number of latent variables, allowing the exploration of chemical and physical transformation pathways via the latent space of the system. The results suggest that the high-veracity imaging data can be used to derive fundamental physical and chemical mechanisms involved, by providing encodings of the observed structures that act as bottom-up equivalents of structural order parameters. The approach also demonstrates the potential of variational (i.e., Bayesian) methods for physical sciences and will stimulate the development of new ways to encode physical constraints in the encoder-decoder architectures, and generative physical laws, topological invariances, and causal relationships in the latent space of VAEs.

Exploring particle dynamics during self-organization processes via rotationally invariant latent representations

The dynamic of complex ordering systems with active rotational degrees of freedom exemplified by protein self-assembly is explored using a machine learning workflow that combines deep learning-based semantic segmentation and rotationally invariant variational autoencoder-based analysis of orientation and shape evolution. The latter allows for disentanglement of the particle orientation from other degrees of freedom and compensates for shifts. The disentangled representations in the latent space encode the rich spectrum of local transitions that can now be visualized and explored via continuous variables. The time dependence of ensemble averages allows insight into the time dynamics of the system, and in particular, illustrates the presence of the potential ordering transition. Finally, analysis of the latent variables along the single-particle trajectory allows tracing these parameters on a single particle level. The proposed approach is expected to be universally applicable for the description of the imaging data in optical, scanning probe, and electron microscopy seeking to understand the dynamics of complex systems where rotations are a significant part of the process.