Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userCan Transfer Neuroevolution Tractably Solve Your Differential Equations?
127
0
0.0
(
0
)
Ask ChatGPT about the research
No Arabic abstract
This paper introduces neuroevolution for solving differential equations. The solution is obtained through optimizing a deep neural network whose loss function is defined by the residual terms from the differential equations. Recent studies have focused on learning such physics-informed neural networks through stochastic gradient descent (SGD) variants, yet they face the difficulty of obtaining an accurate solution due to optimization challenges. In the context of solving differential equations, we are faced with the problem of finding globally optimum parameters of the network, instead of being concerned with out-of-sample generalization. SGD, which searches along a single gradient direction, is prone to become trapped in local optima, so it may not be the best approach here. In contrast, neuroevolution carries out a parallel exploration of diverse solutions with the goal of circumventing local optima. It could potentially find more accurate solutions with better optimized neural networks. However, neuroevolution can be slow, raising tractability issues in practice. With that in mind, a novel and computationally efficient transfer neuroevolution algorithm is proposed in this paper. Our method is capable of exploiting relevant experiential priors when solving a new problem, with adaptation to protect against the risk of negative transfer. The algorithm is applied on a variety of differential equations to empirically demonstrate that transfer neuroevolution can indeed achieve better accuracy and faster convergence than SGD. The experimental outcomes thus establish transfer neuroevolution as a noteworthy approach for solving differential equations, one that has never been studied in the past. Our work expands the resource of available algorithms for optimizing physics-informed neural networks.
rate research
Read More
We study large networks of parametric oscillators as heuristic solvers of random Ising models. In these networks, known as coherent Ising machines, the model to be solved is encoded in the coupling between the oscillators, and a solution is offered by the steady state of the network. This approach relies on the assumption that mode competition steers the network to the ground-state solution of the Ising model. By considering a broad family of frustrated Ising models, we show that the most-efficient mode does not correspond generically to the ground state of the Ising model. We infer that networks of parametric oscillators close to threshold are intrinsically not Ising solvers. Nevertheless, the network can find the correct solution if the oscillators are driven sufficiently above threshold, in a regime where nonlinearities play a predominant role. We find that for all probed instances of the model, the network converges to the ground state of the Ising model with a finite probability.
We propose a physical analogy between finding the solution of an ordinary differential equation (ODE) and a $N$ particle problem in statistical mechanics. It uses the fact that the solution of an ODE is equivalent to obtain the minimum of a functional. Then, we link these two notions, proposing this functional to be the interaction potential energy or thermodynamic potential of an equivalent particle problem. Therefore, solving this statistical mechanics problem amounts to solve the ODE. If only one solution exists, our method provides the unique solution of the ODE. In case we treat an eigenvalue equation, where infinite solutions exist, we obtain the absolute minimum of the corresponding functional or fundamental mode. As a result, it is possible to establish a general relationship between statistical mechanics and ODEs which allows not only to solve them from a physical perspective but also to obtain all relevant thermodynamical equilibrium variables of that particle system related to the differential equation.
Differential equations parameterized by neural networks become expensive to solve numerically as training progresses. We propose a remedy that encourages learned dynamics to be easier to solve. Specifically, we introduce a differentiable surrogate for the time cost of standard numerical solvers, using higher-order derivatives of solution trajectories. These derivatives are efficient to compute with Taylor-mode automatic differentiation. Optimizing this additional objective trades model performance against the time cost of solving the learned dynamics. We demonstrate our approach by training substantially faster, while nearly as accurate, models in supervised classification, density estimation, and time-series modelling tasks.
The structure and performance of neural networks are intimately connected, and by use of evolutionary algorithms, neural network structures optimally adapted to a given task can be explored. Guiding such neuroevolution with additional objectives related to network structure has been shown to improve performance in some cases, especially when modular neural networks are beneficial. However, apart from objectives aiming to make networks more modular, such structural objectives have not been widely explored. We propose two new structural objectives and test their ability to guide evolving neural networks on two problems which can benefit from decomposition into subtasks. The first structural objective guides evolution to align neural networks with a user-recommended decomposition pattern. Intuitively, this should be a powerful guiding target for problems where human users can easily identify a structure. The second structural objective guides evolution towards a population with a high diversity in decomposition patterns. This results in exploration of many different ways to decompose a problem, allowing evolution to find good decompositions faster. Tests on our target problems reveal that both methods perform well on a problem with a very clear and decomposable structure. However, on a problem where the optimal decomposition is less obvious, the structural diversity objective is found to outcompete other structural objectives -- and this technique can even increase performance on problems without any decomposable structure at all.
The iteration sequence based on the BLUES (Beyond Linear Use of Equation Superposition) function method for calculating analytic approximants to solutions of nonlinear ordinary differential equations with sources is elaborated upon. Diverse problems in physics are studied and approximate analytic solutions are found. We first treat a damped driven nonlinear oscillator and show that the method can correctly reproduce oscillatory behaviour. Next, a fractional differential equation describing heat transfer in a semi-infinite rod with Stefan-Boltzmann cooling is handled. In this case, a detailed comparison is made with the Adomian decomposition method, the outcome of which is favourable for the BLUES method. As a final problem, the Fisher equation from population biology is dealt with. For all cases, it is shown that the solutions converge exponentially fast to the numerically exact solution, either globally or, for the Fisher problem, locally.
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
