Challenges in Dynamic Mode Decomposition

Dynamic Mode Decomposition (DMD) is a powerful tool for extracting spatial and temporal patterns from multi-dimensional time series, and it has been used successfully in a wide range of fields, including fluid mechanics, robotics, and neuroscience. Two of the main challenges remaining in DMD research are noise sensitivity and issues related to Krylov space closure when modeling nonlinear systems. Here, we investigate the combination of noise and nonlinearity in a controlled setting, by studying a class of systems with linear latent dynamics which are observed via multinomial observables. Our numerical models include system and measurement noise. We explore the influences of dataset metrics, the spectrum of the latent dynamics, the normality of the system matrix, and the geometry of the dynamics. Our results show that even for these very mildly nonlinear conditions, DMD methods often fail to recover the spectrum and can have poor predictive ability. Our work is motivated by our experience modeling multilegged robot data, where we have encountered great difficulty in reconstructing time series for oscillatory systems with slow transients, which decay only slightly faster than a period.

Predicting the dynamic process and model parameters of the vector optical solitons in birefringent fibers via the modified PINN

A modified physics-informed neural network is used to predict the dynamics of optical pulses including one-soliton, two-soliton, and rogue wave based on the coupled nonlinear Schrodinger equation in birefringent fibers. At the same time, the elastic collision process of the mixed bright-dark soliton is predicted. Compared the predicted results with the exact solution, the modified physics-informed neural network method is proven to be effective to solve the coupled nonlinear Schrodinger equation. Moreover, the dispersion coefficients and nonlinearity coefficients of the coupled nonlinear Schrodinger equation can be learned by modified physics-informed neural network. This provides a reference for us to use deep learning methods to study the dynamic characteristics of solitons in optical fibers.

Modified physics-informed neural network method based on the conservation law constraint and its prediction of optical solitons

Based on conservation laws as one of the important integrable properties of nonlinear physical models, we design a modified physics-informed neural network method based on the conservation law constraint. From a global perspective, this method imposes physical constraints on the solution of nonlinear physical models by introducing the conservation law into the mean square error of the loss function to train the neural network. Using this method, we mainly study the standard nonlinear Schrodinger equation and predict various data-driven optical soliton solutions, including one-soliton, soliton molecules, two-soliton interaction, and rogue wave. In addition, based on various exact solutions, we use the modified physics-informed neural network method based on the conservation law constraint to predict the dispersion and nonlinear coefficients of the standard nonlinear Schrodinger equation. Compared with the traditional physics-informed neural network method, the modified method can significantly improve the calculation accuracy.

Exploiting Language Model for Efficient Linguistic Steganalysis: An Empirical Study

Recent advances in linguistic steganalysis have successively applied CNNs, RNNs, GNNs and other deep learning models for detecting secret information in generative texts. These methods tend to seek stronger feature extractors to achieve higher steganalysis effects. However, we have found through experiments that there actually exists significant difference between automatically generated steganographic texts and carrier texts in terms of the conditional probability distribution of individual words. Such kind of statistical difference can be naturally captured by the language model used for generating steganographic texts, which drives us to give the classifier a priori knowledge of the language model to enhance the steganalysis ability. To this end, we present two methods to efficient linguistic steganalysis in this paper. One is to pre-train a language model based on RNN, and the other is to pre-train a sequence autoencoder. Experimental results show that the two methods have different degrees of performance improvement when compared to the randomly initialized RNN classifier, and the convergence speed is significantly accelerated. Moreover, our methods have achieved the best detection results.

Structural Watermarking to Deep Neural Networks via Network Channel Pruning

In order to protect the intellectual property (IP) of deep neural networks (DNNs), many existing DNN watermarking techniques either embed watermarks directly into the DNN parameters or insert backdoor watermarks by fine-tuning the DNN parameters, which, however, cannot resist against various attack methods that remove watermarks by altering DNN parameters. In this paper, we bypass such attacks by introducing a structural watermarking scheme that utilizes channel pruning to embed the watermark into the host DNN architecture instead of crafting the DNN parameters. To be specific, during watermark embedding, we prune the internal channels of the host DNN with the channel pruning rates controlled by the watermark. During watermark extraction, the watermark is retrieved by identifying the channel pruning rates from the architecture of the target DNN model. Due to the superiority of pruning mechanism, the performance of the DNN model on its original task is reserved during watermark embedding. Experimental results have shown that, the proposed work enables the embedded watermark to be reliably recovered and provides a high watermark capacity, without sacrificing the usability of the DNN model. It is also demonstrated that the work is robust against common transforms and attacks designed for conventional watermarking approaches.

A Mathematical Foundation for Robust Machine Learning based on Bias-Variance Trade-off

A common assumption in machine learning is that samples are independently and identically distributed (i.i.d). However, the contributions of different samples are not identical in training. Some samples are difficult to learn and some samples are noisy. The unequal contributions of samples has a considerable effect on training performances. Studies focusing on unequal sample contributions (e.g., easy, hard, noisy) in learning usually refer to these contributions as robust machine learning (RML). Weighing and regularization are two common techniques in RML. Numerous learning algorithms have been proposed but the strategies for dealing with easy/hard/noisy samples differ or even contradict with different learning algorithms. For example, some strategies take the hard samples first, whereas some strategies take easy first. Conducting a clear comparison for existing RML algorithms in dealing with different samples is difficult due to lack of a unified theoretical framework for RML. This study attempts to construct a mathematical foundation for RML based on the bias-variance trade-off theory. A series of definitions and properties are presented and proved. Several classical learning algorithms are also explained and compared. Improvements of existing methods are obtained based on the comparison. A unified method that combines two classical learning strategies is proposed.

Galois representations, $(varphi, Gamma)$-modules and prismatic F-crystals

We prove that both local Galois representations and $(varphi,Gamma)$-modules can be recovered from prismatic F-crystals, from which we obtain a new proof of the equivalence of Galois representations and $(varphi,Gamma)$-modules.

Improving the Expressive Power of Graph Neural Network with Tinhofer Algorithm

In recent years, Graph Neural Network (GNN) has bloomly progressed for its power in processing graph-based data. Most GNNs follow a message passing scheme, and their expressive power is mathematically limited by the discriminative ability of the Weisfeiler-Lehman (WL) test. Following Tinhofers research on compact graphs, we propose a variation of the message passing scheme, called the Weisfeiler-Lehman-Tinhofer GNN (WLT-GNN), that theoretically breaks through the limitation of the WL test. In addition, we conduct comparative experiments and ablation studies on several well-known datasets. The results show that the proposed methods have comparable performances and better expressive power on these datasets.

Data-driven femtosecond optical soliton excitations and parameters discovery of the high-order NLSE using the PINN

We use the physics-informed neural network to solve a variety of femtosecond optical soliton solutions of the high order nonlinear Schrodinger equation, including one-soliton solution, two-soliton solution, rogue wave solution, W-soliton solution and M-soliton solution. The prediction error for one-soliton, W-soliton and M-soliton is smaller. As the prediction distance increases, the prediction error will gradually increase. The unknown physical parameters of the high order nonlinear Schrodinger equation are studied by using rogue wave solutions as data sets. The neural network is optimized from three aspects including the number of layers of the neural network, the number of neurons, and the sampling points. Compared with previous research, our error is greatly reduced. This is not a replacement for the traditional numerical method, but hopefully to open up new ideas.

Reinforcement Learning Random Access for Delay-Constrained Heterogeneous Wireless Networks: A Two-User Case

In this paper, we investigate the random access problem for a delay-constrained heterogeneous wireless network. As a first attempt to study this new problem, we consider a network with two users who deliver delay-constrained traffic to an access point (AP) via a common unreliable collision wireless channel. We assume that one user (called user 1) adopts ALOHA and we optimize the random access scheme of the other user (called user 2). The most intriguing part of this problem is that user 2 does not know the information of user 1 but needs to maximize the system timely throughput. Such a paradigm of collaboratively sharing spectrum is envisioned by DARPA to better dynamically match the supply and demand in the future [1], [2]. We first propose a Markov Decision Process (MDP) formulation to derive a modelbased upper bound, which can quantify the performance gap of any designed schemes. We then utilize reinforcement learning (RL) to design an R-learning-based [3]-[5] random access scheme, called TSRA. We finally carry out extensive simulations to show that TSRA achieves close-to-upper-bound performance and better performance than the existing baseline DLMA [6], which is our counterpart scheme for delay-unconstrained heterogeneous wireless network. All source code is publicly available in https://github.com/DanzhouWu/TSRA.