Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userAdaptive thresholds for layered neural networks with synaptic noise
64
0
0.0
(
0
)
Ask ChatGPT about the research
No Arabic abstract
The inclusion of a macroscopic adaptive threshold is studied for the retrieval dynamics of layered feedforward neural network models with synaptic noise. It is shown that if the threshold is chosen appropriately as a function of the cross-talk noise and of the activity of the stored patterns, adapting itself automatically in the course of the recall process, an autonomous functioning of the network is guaranteed.This self-control mechanism considerably improves the quality of retrieval, in particular the storage capacity, the basins of attraction and the mutual information content.
rate research
Read More
The inclusion of a macroscopic adaptive threshold is studied for the retrieval dynamics of both layered feedforward and fully connected neural network models with synaptic noise. These two types of architectures require a different method to be solved numerically. In both cases it is shown that, if the threshold is chosen appropriately as a function of the cross-talk noise and of the activity of the stored patterns, adapting itself automatically in the course of the recall process, an autonomous functioning of the network is guaranteed. This self-control mechanism considerably improves the quality of retrieval, in particular the storage capacity, the basins of attraction and the mutual information content.
For the retrieval dynamics of sparsely coded attractor associative memory models with synaptic noise the inclusion of a macroscopic time-dependent threshold is studied. It is shown that if the threshold is chosen appropriately as a function of the cross-talk noise and of the activity of the memorized patterns, adapting itself automatically in the course of the time evolution, an autonomous functioning of the model is guaranteed. This self-control mechanism considerably improves the quality of the fixed-point retrieval dynamics, in particular the storage capacity, the basins of attraction and the mutual information content.
The dynamics of neural networks is often characterized by collective behavior and quasi-synchronous events, where a large fraction of neurons fire in short time intervals, separated by uncorrelated firing activity. These global temporal signals are crucial for brain functioning. They strongly depend on the topology of the network and on the fluctuations of the connectivity. We propose a heterogeneous mean--field approach to neural dynamics on random networks, that explicitly preserves the disorder in the topology at growing network sizes, and leads to a set of self-consistent equations. Within this approach, we provide an effective description of microscopic and large scale temporal signals in a leaky integrate-and-fire model with short term plasticity, where quasi-synchronous events arise. Our equations provide a clear analytical picture of the dynamics, evidencing the contributions of both periodic (locked) and aperiodic (unlocked) neurons to the measurable average signal. In particular, we formulate and solve a global inverse problem of reconstructing the in-degree distribution from the knowledge of the average activity field. Our method is very general and applies to a large class of dynamical models on dense random networks.
We calculate analytically the critical connectivity $K_c$ of Random Threshold Networks (RTN) for homogeneous and inhomogeneous thresholds, and confirm the results by numerical simulations. We find a super-linear increase of $K_c$ with the (average) absolute threshold $|h|$, which approaches $K_c(|h|) sim h^2/(2ln{|h|})$ for large $|h|$, and show that this asymptotic scaling is universal for RTN with Poissonian distributed connectivity and threshold distributions with a variance that grows slower than $h^2$. Interestingly, we find that inhomogeneous distribution of thresholds leads to increased propagation of perturbations for sparsely connected networks, while for densely connected networks damage is reduced; the cross-over point yields a novel, characteristic connectivity $K_d$, that has no counterpart in Boolean networks. Last, local correlations between node thresholds and in-degree are introduced. Here, numerical simulations show that even weak (anti-)correlations can lead to a transition from ordered to chaotic dynamics, and vice versa. It is shown that the naive mean-field assumption typical for the annealed approximation leads to false predictions in this case, since correlations between thresholds and out-degree that emerge as a side-effect strongly modify damage propagation behavior.
The effects of a variable amount of random dilution of the synaptic couplings in Q-Ising multi-state neural networks with Hebbian learning are examined. A fraction of the couplings is explicitly allowed to be anti-Hebbian. Random dilution represents the dying or pruning of synapses and, hence, a static disruption of the learning process which can be considered as a form of multiplicative noise in the learning rule. Both parallel and sequential updating of the neurons can be treated. Symmetric dilution in the statics of the network is studied using the mean-field theory approach of statistical mechanics. General dilution, including asymmetric pruning of the couplings, is examined using the generating functional (path integral) approach of disordered systems. It is shown that random dilution acts as additive gaussian noise in the Hebbian learning rule with a mean zero and a variance depending on the connectivity of the network and on the symmetry. Furthermore, a scaling factor appears that essentially measures the average amount of anti-Hebbian couplings.
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
