Hierarchical deep reinforcement learning controlled three-dimensional navigation of microrobots in blood vessels

Designing intelligent microrobots that can autonomously navigate and perform instructed routines in blood vessels, a complex and crowded environment with obstacles including dense cells, different flow patterns and diverse vascular geometries, can offer enormous possibilities in biomedical applications. Here we report a hierarchical control scheme that enables a microrobot to efficiently navigate and execute customizable routines in blood vessels. The control scheme consists of two highly decoupled components: a high-level controller setting short-ranged dynamic targets to guide the microrobot to follow a preset path and a low-level deep reinforcement learning (DRL) controller responsible for maneuvering microrobots towards these dynamic guiding targets. The proposed DRL controller utilizes three-dimensional (3D) convolutional neural networks and is capable of learning control policy directly from a coarse raw 3D sensory input. In blood vessels with rich configurations of red blood cells and vessel geometry, the control scheme enables efficient navigation and faithful execution of instructed routines. The control scheme is also robust to adversarial perturbations including blood flows. This study provides a proof-of-principle for designing data-driven control systems for autonomous navigation in vascular networks; it illustrates the great potential of artificial intelligence for broad biomedical applications such as target drug delivery, blood clots clear, precision surgery, disease diagnosis, and more.

Constraining early-time dust formation in core-collapse supernovae

There is currently a severe discrepancy between theoretical models of dust formation in core-collapse supernovae (CCSNe), which predict $gtrsim 0.01$ M$_odot$ of ejecta dust forming within $sim 1000$ days, and observations at these epochs, which infer much lower masses. We demonstrate that, in the optically thin case, these low dust masses are robust despite significant observational and model uncertainties. For a sample of 11 well-observed CCSNe, no plausible model reaches carbon dust masses above $10^{-4}$ M$_odot$, or silicate masses above $sim 10^{-3}$ M$_odot$. Optically thick models can accommodate larger dust masses, but the dust must be clumped and have a low ($<0.1$) covering fraction to avoid conflict with data at optical wavelengths. These values are insufficient to reproduce the observed infrared fluxes, and the required covering fraction varies not only between SNe but between epochs for the same object. The difficulty in reconciling large dust masses with early-time observations of CCSNe, combined with well-established detections of comparably large dust masses in supernova remnants, suggests that a mechanism for late-time dust formation is necessary.

Micro/Nano Motor Navigation and Localization via Deep Reinforcement Learning

Efficient navigation and precise localization of Brownian micro/nano self-propelled motor particles within complex landscapes could enable future high-tech applications involving for example drug delivery, precision surgery, oil recovery, and environmental remediation. Here we employ a model-free deep reinforcement learning algorithm based on bio-inspired neural networks to enable different types of micro/nano motors to be continuously controlled to carry out complex navigation and localization tasks. Micro/nano motors with either tunable self-propelling speeds or orientations or both, are found to exhibit strikingly different dynamics. In particular, distinct control strategies are required to achieve effective navigation in free space and obstacle environments, as well as under time constraints. Our findings provide fundamental insights into active dynamics of Brownian particles controlled using artificial intelligence and could guide the design of motor and robot control systems with diverse application requirements.

Efficient Navigation of Colloidal Robots in an Unknown Environment via Deep Reinforcement Learning

Equipping active colloidal robots with intelligence such that they can efficiently navigate in unknown complex environments could dramatically impact their use in emerging applications like precision surgery and targeted drug delivery. Here we develop a model-free deep reinforcement learning that can train colloidal robots to learn effective navigation strategies in unknown environments with random obstacles. We show that trained robot agents learn to make navigation decisions regarding both obstacle avoidance and travel time minimization, based solely on local sensory inputs without prior knowledge of the global environment. Such agents with biologically inspired mechanisms can acquire competitive navigation capabilities in large-scale, complex environments containing obstacles of diverse shapes, sizes, and configurations. This study illustrates the potential of artificial intelligence in engineering active colloidal systems for future applications and constructing complex active systems with visual and learning capability.

A decade of ejecta dust formation in the Type IIn SN 2005ip

In order to understand the contribution of core-collapse supernovae to the dust budget of the early universe, it is important to understand not only the mass of dust that can form in core-collapse supernovae but also the location and rate of dust formation. SN 2005ip is of particular interest since dust has been inferred to have formed in both the ejecta and the post-shock region behind the radiative reverse shock. We have collated eight optical archival spectra that span the lifetime of SN 2005ip and we additionally present a new X-shooter optical-near-IR spectrum of SN 2005ip at 4075d post-discovery. Using the Monte Carlo line transfer code DAMOCLES, we have modelled the blueshifted broad and intermediate width H$alpha$, H$beta$ and He I lines from 48d to 4075d post-discovery using an ejecta dust model. We find that dust in the ejecta can account for the asymmetries observed in the broad and intermediate width H$alpha$, H$beta$ and He I line profiles at all epochs and that it is not necessary to invoke post-shock dust formation to explain the blueshifting observed in the intermediate width post-shock lines. Using a Bayesian approach, we have determined the evolution of the ejecta dust mass in SN 2005ip over 10 years presuming an ejecta dust model, with an increasing dust mass from ~10$^{-8}$ M$_{odot}$ at 48d to a current dust mass of $sim$0.1 M$_{odot}$.

Update of the Unitarity Triangle Analysis

We present the status of the Unitarity Triangle Analysis (UTA), within the Standard Model (SM) and beyond, with experimental and theoretical inputs updated for the ICHEP 2010 conference. Within the SM, we find that the general consistency among all the constraints leaves space only to some tension (between the UTA prediction and the experimental measurement) in BR(B -> tau nu), sin(2 beta) and epsilon_K. In the UTA beyond the SM, we allow for New Physics (NP) effects in (Delta F)=2 processes. The hint of NP at the 2.9 sigma level in the B_s-bar B_s mixing turns out to be confirmed by the present update, which includes the new D0 result on the dimuon charge asymmetry but not the new CDF measurement of phi_s, being the likelihood not yet released.

Combined phase space characterization at the PEP-II IP using single-beam and luminous-region measurements

We present a novel method to characterize the e+/- phase space at the IP of the SLAC B-factory, that combines single-beam measurements with a detailed mapping of luminous-region observables. Transverse spot sizes are determined in the two rings with synchrotron-light monitors and extrapolated to the IP using measured lattice functions. The specific luminosity, which is proportional to the inverse product of the overlap IP beam sizes, is continuously monitored using radiative-Bhabha events. The spatial variation of the luminosity and of the transverse-boost distribution of the colliding e+/-, are measured using e+ e- --> mu+ mu- events reconstructed in the BaBar detector. The combination of these measurements provide constraints on the emittances, horizontal and vertical spot sizes, angular divergences and beta functions of both beams at the IP during physics data-taking. Preliminary results of this combined spot-size analysis are confronted with independent measurements of IP beta-functions and overlap IP beam sizes at low beam current.