Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userBackEISNN: A Deep Spiking Neural Network with Adaptive Self-Feedback and Balanced Excitatory-Inhibitory Neurons
55
0
0.0
(
0
)
Added by
Dongcheng Zhao
Publication date
2021
fields
Informatics Engineering
and research's language is
English
Ask ChatGPT about the research
No Arabic abstract
Spiking neural networks (SNNs) transmit information through discrete spikes, which performs well in processing spatial-temporal information. Due to the non-differentiable characteristic, there still exist difficulties in designing well-performed SNNs. Recently, SNNs trained with backpropagation have shown superior performance due to the proposal of the gradient approximation. However, the performance on complex tasks is still far away from the deep neural networks. Taking inspiration from the autapse in the brain which connects the spiking neurons with a self-feedback connection, we apply an adaptive time-delayed self-feedback on the membrane potential to regulate the spike precisions. As well as, we apply the balanced excitatory and inhibitory neurons mechanism to control the spiking neurons output dynamically. With the combination of the two mechanisms, we propose a deep spiking neural network with adaptive self-feedback and balanced excitatory and inhibitory neurons (BackEISNN). The experimental results on several standard datasets have shown that the two modules not only accelerate the convergence of the network but also improve the accuracy. For the MNIST, FashionMNIST, and N-MNIST datasets, our model has achieved state-of-the-art performance. For the CIFAR10 dataset, our BackEISNN also gets remarkable performance on a relatively light structure that competes against state-of-the-art SNNs.
rate research
Read More
A companion paper introduces a nonlinear network with Hebbian excitatory (E) neurons that are reciprocally coupled with anti-Hebbian inhibitory (I) neurons and also receive Hebbian feedforward excitation from sensory (S) afferents. The present paper derives the network from two normative principles that are mathematically equivalent but conceptually different. The first principle formulates unsupervised learning as a constrained optimization problem: maximization of S-E correlations subject to a copositivity constraint on E-E correlations. A combination of Legendre and Lagrangian duality yields a zero-sum continuous game between excitatory and inhibitory connections that is solved by the neural network. The second principle defines a zero-sum game between E and I cells. E cells want to maximize S-E correlations and minimize E-I correlations, while I cells want to maximize I-E correlations and minimize power. The conflict between I and E objectives effectively forces the E cells to decorrelate from each other, although only incompletely. Legendre duality yields the neural network.
Dynamic excitatory-inhibitory (E-I) balance is a paradigmatic mechanism invoked to explain the irregular low firing activity observed in the cortex. However, we will show that the E-I balance can be at the origin of other regimes observable in the brain. The analysis is performed by combining simulations of sparse E-I networks composed of N spiking neurons with analytical investigations of low dimensional neural mass models. The bifurcation diagrams, derived for the neural mass model, allow to classify the asynchronous and coherent behaviours emerging any finite in-degree K. In the limit N >> K >> 1 both supra and sub-threshold balanced asynchronous regimes can be observed. Due to structural heterogeneity the asynchronous states are characterized by the splitting of the neurons in three groups: silent, fluctuation and mean driven. The coherent rhythms are characterized by regular or irregular temporal fluctuations joined to spatial coherence similar to coherent fluctuations observed in the cortex over multiple spatial scales. Collective Oscillations (COs) can emerge due to two different mechanisms. A first mechanism similar to the pyramidal-interneuron gamma (PING) one. The second mechanism is intimately related to the presence of current fluctuations, which sustain COs characterized by an essentially simultaneous bursting of the two populations. We observe period-doubling cascades involving the PING-like COs finally leading to the appearance of coherent chaos. For sufficiently strong current fluctuations we report a novel mechanism of frequency locking among collective rhythms promoted by these intrinsic fluctuations. Our analysis suggest that despite PING-like or fluctuation driven COS are observable for any finite in-degree K, in the limit N >> K >> 1 these solutions result in two coexisting balanced regimes: an asynchronous and a fully synchronized one.
Collective oscillations and their suppression by external stimulation are analyzed in a large-scale neural network consisting of two interacting populations of excitatory and inhibitory quadratic integrate-and-fire neurons. In the limit of an infinite number of neurons, the microscopic model of this network can be reduced to an exact low-dimensional system of mean-field equations. Bifurcation analysis of these equations reveals three different dynamic modes in a free network: a stable resting state, a stable limit cycle, and bistability with a coexisting resting state and a limit cycle. We show that in the limit cycle mode, high-frequency stimulation of an inhibitory population can stabilize an unstable resting state and effectively suppress collective oscillations. We also show that in the bistable mode, the dynamics of the network can be switched from a stable limit cycle to a stable resting state by applying an inhibitory pulse to the excitatory population. The results obtained from the mean-field equations are confirmed by numerical simulation of the microscopic model.
It is widely appreciated that well-balanced excitation and inhibition are necessary for proper function in neural networks. However, in principle, such balance could be achieved by many possible configurations of excitatory and inhibitory strengths, and relative numbers of excitatory and inhibitory neurons. For instance, a given level of excitation could be balanced by either numerous inhibitory neurons with weak synapses, or few inhibitory neurons with strong synapses. Among the continuum of different but balanced configurations, why should any particular configuration be favored? Here we address this question in the context of the entropy of network dynamics by studying an analytically tractable network of binary neurons. We find that entropy is highest at the boundary between excitation-dominant and inhibition-dominant regimes. Entropy also varies along this boundary with a trade-off between high and robust entropy: weak synapse strengths yield high network entropy which is fragile to parameter variations, while strong synapse strengths yield a lower, but more robust, network entropy. In the case where inhibitory and excitatory synapses are constrained to have similar strength, we find that a small, but non-zero fraction of inhibitory neurons, like that seen in mammalian cortex, results in robust and relatively high entropy.
Spiking neural networks (SNNs) has attracted much attention due to its great potential of modeling time-dependent signals. The firing rate of spiking neurons is decided by control rate which is fixed manually in advance, and thus, whether the firing rate is adequate for modeling actual time series relies on fortune. Though it is demanded to have an adaptive control rate, it is a non-trivial task because the control rate and the connection weights learned during the training process are usually entangled. In this paper, we show that the firing rate is related to the eigenvalue of the spike generation function. Inspired by this insight, by enabling the spike generation function to have adaptable eigenvalues rather than parametric control rates, we develop the Bifurcation Spiking Neural Network (BSNN), which has an adaptive firing rate and is insensitive to the setting of control rates. Experiments validate the effectiveness of BSNN on a broad range of tasks, showing that BSNN achieves superior performance to existing SNNs and is robust to the setting of control rates.
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
