We introduce in this work the normalizing field flows (NFF) for learning random fields from scattered measurements. More precisely, we construct a bijective transformation (a normalizing flow characterizing by neural networks) between a Gaussian rand
om field with the Karhunen-Lo`eve (KL) expansion structure and the target stochastic field, where the KL expansion coefficients and the invertible networks are trained by maximizing the sum of the log-likelihood on scattered measurements. This NFF model can be used to solve data-driven forward, inverse, and mixed forward/inverse stochastic partial differential equations in a unified framework. We demonstrate the capability of the proposed NFF model for learning Non Gaussian processes and different types of stochastic partial differential equations.
RGB-D saliency detection has attracted increasing attention, due to its effectiveness and the fact that depth cues can now be conveniently captured. Existing works often focus on learning a shared representation through various fusion strategies, wit
h few methods explicitly considering how to preserve modality-specific characteristics. In this paper, taking a new perspective, we propose a specificity-preserving network (SP-Net) for RGB-D saliency detection, which benefits saliency detection performance by exploring both the shared information and modality-specific properties (e.g., specificity). Specifically, two modality-specific networks and a shared learning network are adopted to generate individual and shared saliency maps. A cross-enhanced integration module (CIM) is proposed to fuse cross-modal features in the shared learning network, which are then propagated to the next layer for integrating cross-level information. Besides, we propose a multi-modal feature aggregation (MFA) module to integrate the modality-specific features from each individual decoder into the shared decoder, which can provide rich complementary multi-modal information to boost the saliency detection performance. Further, a skip connection is used to combine hierarchical features between the encoder and decoder layers. Experiments on six benchmark datasets demonstrate that our SP-Net outperforms other state-of-the-art methods. Code is available at: https://github.com/taozh2017/SPNet.
Photo-realistic facial video portrait reenactment benefits virtual production and numerous VR/AR experiences. The task remains challenging as the portrait should maintain high realism and consistency with the target environment. In this paper, we pre
sent a relightable neural video portrait, a simultaneous relighting and reenactment scheme that transfers the head pose and facial expressions from a source actor to a portrait video of a target actor with arbitrary new backgrounds and lighting conditions. Our approach combines 4D reflectance field learning, model-based facial performance capture and target-aware neural rendering. Specifically, we adopt a rendering-to-video translation network to first synthesize high-quality OLAT imagesets and alpha mattes from hybrid facial performance capture results. We then design a semantic-aware facial normalization scheme to enable reliable explicit control as well as a multi-frame multi-task learning strategy to encode content, segmentation and temporal information simultaneously for high-quality reflectance field inference. After training, our approach further enables photo-realistic and controllable video portrait editing of the target performer. Reliable face poses and expression editing is obtained by applying the same hybrid facial capture and normalization scheme to the source video input, while our explicit alpha and OLAT output enable high-quality relit and background editing. With the ability to achieve simultaneous relighting and reenactment, we are able to improve the realism in a variety of virtual production and video rewrite applications.
This paper is concerned with a novel deep learning method for variational problems with essential boundary conditions. To this end, we first reformulate the original problem into a minimax problem corresponding to a feasible augmented Lagrangian, whi
ch can be solved by the augmented Lagrangian method in an infinite dimensional setting. Based on this, by expressing the primal and dual variables with two individual deep neural network functions, we present an augmented Lagrangian deep learning method for which the parameters are trained by the stochastic optimization method together with a projection technique. Compared to the traditional penalty method, the new method admits two main advantages: i) the choice of the penalty parameter is flexible and robust, and ii) the numerical solution is more accurate in the same magnitude of computational cost. As typical applications, we apply the new approach to solve elliptic problems and (nonlinear) eigenvalue problems with essential boundary conditions, and numerical experiments are presented to show the effectiveness of the new method.
A variety of polymeric surfaces, such as anti-corrosion coatings and polymer-modified asphalts, are prone to blistering when exposed to moisture and air. As water and oxygen diffuse through the material, dissolved species are produced, which generate
osmotic pressure that deforms and debonds the coating.These mechanisms are experimentally well-supported; however, comprehensive macroscopic models capable of predicting the formation osmotic blisters, without extensive data-fitting, is scant. Here, we develop a general mathematical theory of blistering and apply it to the failure of anti-corrosion coatings on carbon steel. The model is able to predict the irreversible, nonlinear blister growth dynamics, which eventually reaches a stable state, ruptures, or undergoes runaway delamination, depending on the mechanical and adhesion properties of the coating. For runaway delamination, the theory predicts a critical delamination length, beyond which unstable corrosion-driven growth occurs. The model is able to fit multiple sets of blister growth data with no fitting parameters. Corrosion experiments are also performed to observe undercoat rusting on carbon steel, which yielded trends comparable with model predictions. The theory is used to define three dimensionless numbers which can be used for engineering design of elastic coatings capable of resisting visible deformation, rupture, and delamination.
We study the topological properties of the nodal-line semimetal superconductor. The single band inversion and the double band inversion coexist in an $s$-wave nodal-line semimetal superconductor. In the single/double band inversion region, the system
is in a stable/fragile topological state. The two topological invariants describing these two topological states are coupled to each other, leading to the coupled edge states. The stable topological state is indexed by ${mathrm Z}$(d=1), while the fragile topological state is characterized to be ${mathrm Z}otimes {mathrm Z}(d=1)$. In addition, the $s$-wave nodal-line semimetal superconductor has a nontrivial ${mathrm Z_{4}=2}$ topological invariant, indicating that it is a inversion symmetry protected second order topological crystalline superconductor. While the $p$-wave nodal-line semimetal belongs to a pure fragile topological superconductor due to the double band inversion. The vortex bound states and the surface impurity effects are studied and they can be used to distinguish the different pairing states and identify the fragile topology of the system. Remarkably, we propose that vortex line in the nodal-line semimetal superconductor is a one dimensional fragile topological state protected by the spatial symmetry.
Camouflaged object detection (COD) is a challenging task due to the low boundary contrast between the object and its surroundings. In addition, the appearance of camouflaged objects varies significantly, e.g., object size and shape, aggravating the d
ifficulties of accurate COD. In this paper, we propose a novel Context-aware Cross-level Fusion Network (C2F-Net) to address the challenging COD task. Specifically, we propose an Attention-induced Cross-level Fusion Module (ACFM) to integrate the multi-level features with informative attention coefficients. The fused features are then fed to the proposed Dual-branch Global Context Module (DGCM), which yields multi-scale feature representations for exploiting rich global context information. In C2F-Net, the two modules are conducted on high-level features using a cascaded manner. Extensive experiments on three widely used benchmark datasets demonstrate that our C2F-Net is an effective COD model and outperforms state-of-the-art models remarkably. Our code is publicly available at: https://github.com/thograce/C2FNet.
Model protection is vital when deploying Convolutional Neural Networks (CNNs) for commercial services, due to the massive costs of training them. In this work, we propose a selective encryption (SE) algorithm to protect CNN models from unauthorized a
ccess, with a unique feature of providing hierarchical services to users. Our algorithm firstly selects important model parameters via the proposed Probabilistic Selection Strategy (PSS). It then encrypts the most important parameters with the designed encryption method called Distribution Preserving Random Mask (DPRM), so as to maximize the performance degradation by encrypting only a very small portion of model parameters. We also design a set of access permissions, using which different amounts of the most important model parameters can be decrypted. Hence, different levels of model performance can be naturally provided for users. Experimental results demonstrate that the proposed scheme could effectively protect the classification model VGG19 by merely encrypting 8% parameters of convolutional layers. We also implement the proposed model protection scheme in the denoising model DnCNN, showcasing the hierarchical denoising services
The increasing data availability and imported analyzing tools from computer science and physical science have sharply changed traditional methodologies of social sciences, leading to a new branch named computational socioeconomics that studies variou
s phenomena in socioeconomic development by using quantitative methods based on large-scale real-world data. Sited on recent publications, this Perspective will introduce three representative methods: (i) natural data analyses, (ii) large-scale online experiments, and (iii) integration of big data and surveys. This Perspective ends up with in-depth discussion on the limitations and challenges of the above-mentioned emerging methods.
Modern scientific research has become largely a cooperative activity in the Internet age. We build a simulation model to understand the population-level creativity based on the heuristic ant colony algorithm. Each researcher has two heuristic paramet
ers characterizing the goodness of his own judgments and his trust on literature. In a population with all kinds of researchers, we find that as the problem scale increases, the contributor distribution significantly shifts from the independent regime of relying on ones own judgments to the cooperative regime of more closely following the literature. The distribution also changes with the stage of the research problem and the computing power available. Our work provides some preliminary understanding and guidance for the dynamical process of cooperative scientific research in various disciplines.
