Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userEnhanced storage capacity with errors in scale-free Hopfield neural networks: an analytical study
51
0
0.0
(
0
)
Ask ChatGPT about the research
No Arabic abstract
The Hopfield model is a pioneering neural network model with associative memory retrieval. The analytical solution of the model in mean field limit revealed that memories can be retrieved without any error up to a finite storage capacity of $O(N)$, where $N$ is the system size. Beyond the threshold, they are completely lost. Since the introduction of the Hopfield model, the theory of neural networks has been further developed toward realistic neural networks using analog neurons, spiking neurons, etc. Nevertheless, those advances are based on fully connected networks, which are inconsistent with recent experimental discovery that the number of connections of each neuron seems to be heterogeneous, following a heavy-tailed distribution. Motivated by this observation, we consider the Hopfield model on scale-free networks and obtain a different pattern of associative memory retrieval from that obtained on the fully connected network: the storage capacity becomes tremendously enhanced but with some error in the memory retrieval, which appears as the heterogeneity of the connections is increased. Moreover, the error rates are also obtained on several real neural networks and are indeed similar to that on scale-free model networks.
rate research
Read More
We propose a new framework to understand how quantum effects may impact on the dynamics of neural networks. We implement the dynamics of neural networks in terms of Markovian open quantum systems, which allows us to treat thermal and quantum coherent effects on the same footing. In particular, we propose an open quantum generalisation of the celebrated Hopfield neural network, the simplest toy model of associative memory. We determine its phase diagram and show that quantum fluctuations give rise to a qualitatively new non-equilibrium phase. This novel phase is characterised by limit cycles corresponding to high-dimensional stationary manifolds that may be regarded as a generalisation of storage patterns to the quantum domain.
We study the problem of determining the capacity of the binary perceptron for two variants of the problem where the corresponding constraint is symmetric. We call these variants the rectangle-binary-perceptron (RPB) and the $u-$function-binary-perceptron (UBP). We show that, unlike for the usual step-function-binary-perceptron, the critical capacity in these symmetric cases is given by the annealed computation in a large region of parameter space (for all rectangular constraints and for narrow enough $u-$function constraints, $K<K^*$). We prove this fact (under two natural assumptions) using the first and second moment methods. We further use the second moment method to conjecture that solutions of the symmetric binary perceptrons are organized in a so-called frozen-1RSB structure, without using the replica method. We then use the replica method to estimate the capacity threshold for the UBP case when the $u-$function is wide $K>K^*$. We conclude that full-step-replica-symmetry breaking would have to be evaluated in order to obtain the exact capacity in this case.
Macroscopic spin ensembles possess brain-like features such as non-linearity, plasticity, stochasticity, selfoscillations, and memory effects, and therefore offer opportunities for neuromorphic computing by spintronics devices. Here we propose a physical realization of artificial neural networks based on magnetic textures, which can update their weights intrinsically via built-in physical feedback utilizing the plasticity and large number of degrees of freedom of the magnetic domain patterns and without resource-demanding external computations. We demonstrate the idea by simulating the operation of a 4-node Hopfield neural network for pattern recognition.
We investigate analytically and numerically the critical line in undirected random Boolean networks with arbitrary degree distributions, including scale-free topology of connections $P(k)sim k^{-gamma}$. We show that in infinite scale-free networks the transition between frozen and chaotic phase occurs for $3<gamma < 3.5$. The observation is interesting for two reasons. First, since most of critical phenomena in scale-free networks reveal their non-trivial character for $gamma<3$, the position of the critical line in Kauffman model seems to be an important exception from the rule. Second, since gene regulatory networks are characterized by scale-free topology with $gamma<3$, the observation that in finite-size networks the mentioned transition moves towards smaller $gamma$ is an argument for Kauffman model as a good starting point to model real systems. We also explain that the unattainability of the critical line in numerical simulations of classical random graphs is due to percolation phenomena.
Scale-free networks with topology-dependent interactions are studied. It is shown that the universality classes of critical behavior, which conventionally depend only on topology, can also be explored by tuning the interactions. A mapping, $gamma = (gamma - mu)/(1-mu)$, describes how a shift of the standard exponent $gamma$ of the degree distribution $P(q)$ can absorb the effect of degree-dependent pair interactions $J_{ij} propto (q_iq_j)^{-mu}$. Replica technique, cavity method and Monte Carlo simulation support the physical picture suggested by Landau theory for the critical exponents and by the Bethe-Peierls approximation for the critical temperature. The equivalence of topology and interaction holds for equilibrium and non-equilibrium systems, and is illustrated with interdisciplinary applications.
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
