Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userNon-line-of-sight imaging off a Phong surface through deep learning
372
0
0.0
(
0
)
Authors
Chen Zhou
Ask ChatGPT about the research
No Arabic abstract
A deep learning based non-line-of-sight (NLOS) imaging system is developed to image an occluded object off a scattering surface. The neural net is trained using only handwritten digits, and yet exhibits capability to reconstruct patterns distinct from the training set, including physical objects. It can also reconstruct a cartoon video from its scattering patterns in real time, demonstrating the robustness and generalization capability of the deep learning based approach. Several scattering surfaces with varying degree of Lambertian and specular contributions were examined experimentally; it is found that for a Lambertian surface the structural similarity index (SSIM) of reconstructed images is about 0.63, while the SSIM obtained from a scattering surface possessing a specular component can be as high as 0.93. A forward model of light transport was developed based on the Phong scattering model. Scattering patterns from Phong surfaces with different degrees of specular contribution were numerically simulated. It is found that a specular contribution of as small as 5% can enhance the SSIM from 0.83 to 0.93, consistent with the results from experimental data. Singular value spectra of the underlying transfer matrix were calculated for various Phong surfaces. As the weight and the shininess factor increase, i.e., the specular contribution increases, the singular value spectrum broadens and the 50-dB bandwidth is increased by more than 4X with a 10% specular contribution, which indicates that at the presence of even a small amount of specular contribution the NLOS measurement can retain significantly more singular value components, leading to higher reconstruction fidelity. With an ordinary camera and incoherent light source, this work enables a low-cost, real-time NLOS imaging system without the need of an explicit physical model of the underlying light transport process.
rate research
Read More
We develop a scannerless non-line-of-sight three dimensional imaging system based on a commercial 32x32 SPAD camera combined with a 70 ps pulsed laser. In our experiment, 1024 time histograms can be achieved synchronously in 3s with an average time resolution of about 165 ps. The result with filtered back projection shows a discernable reconstruction while the result using virtual wave field demonstrates a better quality similar to the ones created by earlier scanning imaging systems with single pixel SPAD. Comparatively, our system has large potential advantages in frame frequency, power requirements, compactness and robustness. The research results will pave a path for scannerless non-line-of-sight three dimensional imaging application.
We present a neural modeling framework for Non-Line-of-Sight (NLOS) imaging. Previous solutions have sought to explicitly recover the 3D geometry (e.g., as point clouds) or voxel density (e.g., within a pre-defined volume) of the hidden scene. In contrast, inspired by the recent Neural Radiance Field (NeRF) approach, we use a multi-layer perceptron (MLP) to represent the neural transient field or NeTF. However, NeTF measures the transient over spherical wavefronts rather than the radiance along lines. We therefore formulate a spherical volume NeTF reconstruction pipeline, applicable to both confocal and non-confocal setups. Compared with NeRF, NeTF samples a much sparser set of viewpoints (scanning spots) and the sampling is highly uneven. We thus introduce a Monte Carlo technique to improve the robustness in the reconstruction. Comprehensive experiments on synthetic and real datasets demonstrate NeTF provides higher quality reconstruction and preserves fine details largely missing in the state-of-the-art.
We consider the non-line-of-sight (NLOS) imaging of an object using the light reflected off a diffusive wall. The wall scatters incident light such that a lens is no longer useful to form an image. Instead, we exploit the 4D spatial coherence function to reconstruct a 2D projection of the obscured object. The approach is completely passive in the sense that no control over the light illuminating the object is assumed and is compatible with the partially coherent fields ubiquitous in both the indoor and outdoor environments. We formulate a multi-criteria convex optimization problem for reconstruction, which fuses the reflected fields intensity and spatial coherence information at different scales. Our formulation leverages established optics models of light propagation and scattering and exploits the sparsity common to many images in different bases. We also develop an algorithm based on the alternating direction method of multipliers to efficiently solve the convex program proposed. A means for analyzing the null space of the measurement matrices is provided as well as a means for weighting the contribution of individual measurements to the reconstruction. This paper holds promise to advance passive imaging in the challenging NLOS regimes in which the intensity does not necessarily retain distinguishable features and provides a framework for multi-modal information fusion for efficient scene reconstruction.
Emerging single-photon-sensitive sensors combined with advanced inverse methods to process picosecond-accurate time-stamped photon counts have given rise to unprecedented imaging capabilities. Rather than imaging photons that travel along direct paths from a source to an object and back to the detector, non-line-of-sight (NLOS) imaging approaches analyse photons {scattered from multiple surfaces that travel} along indirect light paths to estimate 3D images of scenes outside the direct line of sight of a camera, hidden by a wall or other obstacles. Here we review recent advances in the field of NLOS imaging, discussing how to see around corners and future prospects for the field.
Non-line-of-sight (NLOS) imaging is based on capturing the multi-bounce indirect reflections from the hidden objects. Active NLOS imaging systems rely on the capture of the time of flight of light through the scene, and have shown great promise for the accurate and robust reconstruction of hidden scenes without the need for specialized scene setups and prior assumptions. Despite that existing methods can reconstruct 3D geometries of the hidden scene with excellent depth resolution, accurately recovering object textures and appearance with high lateral resolution remains an challenging problem. In this work, we propose a new problem formulation, called NLOS photography, to specifically address this deficiency. Rather than performing an intermediate estimate of the 3D scene geometry, our method follows a data-driven approach and directly reconstructs 2D images of a NLOS scene that closely resemble the pictures taken with a conventional camera from the location of the relay wall. This formulation largely simplifies the challenging reconstruction problem by bypassing the explicit modeling of 3D geometry, and enables the learning of a deep model with a relatively small training dataset. The results are NLOS reconstructions of unprecedented lateral resolution and image quality.
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
