This paper focuses on a new problem of estimating human pose and shape from single polarization images. Polarization camera is known to be able to capture the polarization of reflected lights that preserves rich geometric cues of an object surface. I
nspired by the recent applications in surface normal reconstruction from polarization images, in this paper, we attempt to estimate human pose and shape from single polarization images by leveraging the polarization-induced geometric cues. A dedicated two-stage pipeline is proposed: given a single polarization image, stage one (Polar2Normal) focuses on the fine detailed human body surface normal estimation; stage two (Polar2Shape) then reconstructs clothed human shape from the polarization image and the estimated surface normal. To empirically validate our approach, a dedicated dataset (PHSPD) is constructed, consisting of over 500K frames with accurate pose and shape annotations. Empirical evaluations on this real-world dataset as well as a synthetic dataset, SURREAL, demonstrate the effectiveness of our approach. It suggests polarization camera as a promising alternative to the more conventional RGB camera for human pose and shape estimation.
Event camera is an emerging imaging sensor for capturing dynamics of moving objects as events, which motivates our work in estimating 3D human pose and shape from the event signals. Events, on the other hand, have their unique challenges: rather than
capturing static body postures, the event signals are best at capturing local motions. This leads us to propose a two-stage deep learning approach, called EventHPE. The first-stage, FlowNet, is trained by unsupervised learning to infer optical flow from events. Both events and optical flow are closely related to human body dynamics, which are fed as input to the ShapeNet in the second stage, to estimate 3D human shapes. To mitigate the discrepancy between image-based flow (optical flow) and shape-based flow (vertices movement of human body shape), a novel flow coherence loss is introduced by exploiting the fact that both flows are originated from the identical human motion. An in-house event-based 3D human dataset is curated that comes with 3D pose and shape annotations, which is by far the largest one to our knowledge. Empirical evaluations on DHP19 dataset and our in-house dataset demonstrate the effectiveness of our approach.
We compute the quantum cohomology of symplectic flag manifolds. Symplectic flag manifolds can be described by non-abelian GLSMs with superpotential. Although the ring relations cannot be directly read off from the equations of motion on the Coulomb b
ranch due to complication introduced by the non-abelian gauge symmetry, it can be shown that they can be extracted from the localization formula in a gauge-invariant form. Our result is general for all symplectic flag manifolds, which reduces to previously established results on symplectic Grassmannians and complete symplectic flag manifolds derived by other means. We also explain why a (0,2) deformation of the GLSM does not give rise to a deformation of the quantum cohomology.
Clays and micas are receiving attention as materials that, in their atomically thin form, could allow for novel proton conductive, ion selective, osmotic power generation, or solvent filtration membranes. The interest arises from the possibility of c
ontrolling their properties by exchanging ions in the crystal lattice. However, the ion exchange process itself remains largely unexplored in atomically thin materials. Here we use atomic-resolution scanning transmission electron microscopy to study the dynamics of the process and reveal the binding sites of individual ions in atomically thin and artificially restacked clays and micas. Imaging ion exchange after different exposure time and for different crystal thicknesses, we find that the ion diffusion constant, D, for the interlayer space of atomically thin samples is up to 10^4 times larger than in bulk crystals and approaches its value in free water. Surprisingly, samples where no bulk exchange is expected display fast exchange if the mica layers are twisted and restacked; but in this case, the exchanged ions arrange in islands controlled by the moire superlattice dimensions. We attribute the fast ion diffusion to enhanced interlayer expandability resulting from weaker interlayer binding forces in both atomically thin and restacked materials. Finally, we demonstrate images of individual surface cations for these materials, which had remained elusive in previous studies. This work provides atomic scale insights into ion diffusion in highly confined spaces and suggests strategies to design novel exfoliated clays membranes.
The flexibility of the bacterial flagellar hook is believed to have substantial consequences for microorganism locomotion. Using a simplified model of a rigid flagellum and a flexible hook, we show that the paths of axisymmetric cell bodies driven by
a single flagellum in Stokes flow are generically helical. Phase-averaged resistance and mobility tensors are produced to describe the flagellar hydrodynamics, and a helical rod model which retains a coupling between translation and rotation is identified as a distinguished asymptotic limit. A supercritical Hopf bifurcation in the flagellar orientation beyond a critical ratio of flagellar motor torque to hook bending stiffness, which is set by the spontaneous curvature of the flexible hook, the shape of the cell body, and the flagellum geometry, can have a dramatic effect on the cells trajectory through the fluid. Although the equilibrium hook angle can result in a wide variance in the trajectorys helical pitch, we find a very consistent prediction for the trajectorys helical amplitude using parameters relevant to swimming P. aeruginosa cells.
The convergent beam electron diffraction (CBED) patterns of twisted bilayer samples exhibit interference patterns in their CBED spots. Such interference patterns can be treated as off-axis holograms and the phase of the scattered waves, meaning the i
nterlayer distance can be reconstructed. A detailed protocol of the reconstruction procedure is provided in this study. In addition, we derive an exact formula for reconstructing the interlayer distance from the recovered phase distribution, which takes into account the different chemical compositions of the individual monolayers. It is shown that one interference fringe in a CBED spot is sufficient to reconstruct the distance between the layers, which can be practical for imaging samples with a relatively small twist angle or when probing small sample regions. The quality of the reconstructed interlayer distance is studied as a function of the twist angle. At smaller twist angles, the reconstructed interlayer distance distribution is more precise and artefact free. At larger twist angles, artefacts due to the moire structure appear in the reconstruction. A method for the reconstruction of the average interlayer distance is presented. As for resolution, the interlayer distance can be reconstructed by the holographic approach at an accuracy of 0.5 A, which is a few hundred times better than the intrinsic z-resolution of diffraction limited resolution, as expressed through the spread of the measured k-values. Moreover, we show that holographic CBED imaging can detect variations as small as 0.1 A in the interlayer distance, though the quantitative reconstruction of such variations suffers from large errors.
This paper tackles the problem of estimating 3D body shape of clothed humans from single polarized 2D images, i.e. polarization images. Polarization images are known to be able to capture polarized reflected lights that preserve rich geometric cues o
f an object, which has motivated its recent applications in reconstructing surface normal of the objects of interest. Inspired by the recent advances in human shape estimation from single color images, in this paper, we attempt at estimating human body shapes by leveraging the geometric cues from single polarization images. A dedicated two-stage deep learning approach, SfP, is proposed: given a polarization image, stage one aims at inferring the fined-detailed body surface normal; stage two gears to reconstruct the 3D body shape of clothing details. Empirical evaluations on a synthetic dataset (SURREAL) as well as a real-world dataset (PHSPD) demonstrate the qualitative and quantitative performance of our approach in estimating human poses and shapes. This indicates polarization camera is a promising alternative to the more conventional color or depth imaging for human shape estimation. Further, normal maps inferred from polarization imaging play a significant role in accurately recovering the body shapes of clothed people.
Polarization images are known to be able to capture polarized reflected lights that preserve rich geometric cues of an object, which has motivated its recent applications in reconstructing detailed surface normal of the objects of interest. Meanwhile
, inspired by the recent breakthroughs in human shape estimation from a single color image, we attempt to investigate the new question of whether the geometric cues from polarization camera could be leveraged in estimating detailed human body shapes. This has led to the curation of Polarization Human Shape and Pose Dataset (PHSPD), our home-grown polarization image dataset of various human shapes and poses.
In complex environments, low-cost and robust localization is a challenging problem. For example, in a GPSdenied environment, LiDAR can provide accurate position information, but the cost is high. In general, visual SLAM based localization methods bec
ome unreliable when the sunlight changes greatly. Therefore, inexpensive and reliable methods are required. In this paper, we propose a stereo visual localization method based on the prior LiDAR map. Different from the conventional visual localization system, we design a novel visual optimization model by matching planar information between the LiDAR map and visual image. Bundle adjustment is built by using coplanarity constraints. To solve the optimization problem, we use a graph-based optimization algorithm and a local window optimization method. Finally, we estimate a full six degrees of freedom (DOF) pose without scale drift. To validate the efficiency, the proposed method has been tested on the KITTI dataset. The results show that our method is more robust and accurate than the state-of-art ORB-SLAM2.
Van der Waals heterostructures form a massive interdisciplinary research field, fueled by the rich material science opportunities presented by layer assembly of artificial solids with controlled composition, order and relative rotation of adjacent at
omic planes. Here we use atomic resolution transmission electron microscopy and multiscale modeling to show that the lattice of MoS$_2$ and WS$_2$ bilayers twisted to a small angle, $theta<3^{circ}$, reconstructs into energetically favorable stacking domains separated by a network of stacking faults. For crystal alignments close to 3R stacking, a tessellated pattern of mirror reflected triangular 3R domains emerges, separated by a network of partial dislocations which persist to the smallest twist angles. Scanning tunneling measurements show that the electronic properties of those 3R domains appear qualitatively different from 2H TMDs, featuring layer-polarized conduction band states caused by lack of both inversion and mirror symmetry. In contrast, for alignments close to 2H stacking, stable 2H domains dominate, with nuclei of an earlier unnoticed metastable phase limited to $sim$ 5nm in size. This appears as a kagome-like pattern at $thetasim 1^{circ}$, transitioning at $thetarightarrow 0$ to a hexagonal array of screw dislocations separating large-area 2H domains.
