Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userSIMBA: A Skyrmionic In-Memory Binary Neural Network Accelerator
87
0
0.0
(
0
)
Ask ChatGPT about the research
No Arabic abstract
Magnetic skyrmions are emerging as potential candidates for next generation non-volatile memories. In this paper, we propose an in-memory binary neural network (BNN) accelerator based on the non-volatile skyrmionic memory, which we call as SIMBA. SIMBA consumes 26.7 mJ of energy and 2.7 ms of latency when running an inference on a VGG-like BNN. Furthermore, we demonstrate improvements in the performance of SIMBA by optimizing material parameters such as saturation magnetization, anisotropic energy and damping ratio. Finally, we show that the inference accuracy of BNNs is robust against the possible stochastic behavior of SIMBA (88.5% +/- 1%).
rate research
Read More
Neural networks span a wide range of applications of industrial and commercial significance. Binary neural networks (BNN) are particularly effective in trading accuracy for performance, energy efficiency or hardware/software complexity. Here, we introduce a spintronic, re-configurable in-memory BNN accelerator, PIMBALL: Processing In Memory BNN AcceL(L)erator, which allows for massively parallel and energy efficient computation. PIMBALL is capable of being used as a standard spintronic memory (STT-MRAM) array and a computational substrate simultaneously. We evaluate PIMBALL using multiple image classifiers and a genomics kernel. Our simulation results show that PIMBALL is more energy efficient than alternative CPU, GPU, and FPGA based implementations while delivering higher throughput.
There is increasing demand to bring machine learning capabilities to low power devices. By integrating the computational power of machine learning with the deployment capabilities of low power devices, a number of new applications become possible. In some applications, such devices will not even have a battery, and must rely solely on energy harvesting techniques. This puts extreme constraints on the hardware, which must be energy efficient and capable of tolerating interruptions due to power outages. Here, as a representative example, we propose an in-memory support vector machine learning accelerator utilizing non-volatile spintronic memory. The combination of processing-in-memory and non-volatility provides a key advantage in that progress is effectively saved after every operation. This enables instant shut down and restart capabilities with minimal overhead. Additionally, the operations are highly energy efficient leading to low power consumption.
In-memory computing is a promising non-von Neumann approach for making energy-efficient deep learning inference hardware. Crossbar arrays of resistive memory devices can be used to encode the network weights and perform efficient analog matrix-vector multiplications without intermediate movements of data. However, due to device variability and noise, the network needs to be trained in a specific way so that transferring the digitally trained weights to the analog resistive memory devices will not result in significant loss of accuracy. Here, we introduce a methodology to train ResNet-type convolutional neural networks that results in no appreciable accuracy loss when transferring weights to in-memory computing hardware based on phase-change memory (PCM). We also propose a compensation technique that exploits the batch normalization parameters to improve the accuracy retention over time. We achieve a classification accuracy of 93.7% on the CIFAR-10 dataset and a top-1 accuracy on the ImageNet benchmark of 71.6% after mapping the trained weights to PCM. Our hardware results on CIFAR-10 with ResNet-32 demonstrate an accuracy above 93.5% retained over a one day period, where each of the 361,722 synaptic weights of the network is programmed on just two PCM devices organized in a differential configuration.
The memristive crossbar aims to implement analog weighted neural network, however, the realistic implementation of such crossbar arrays is not possible due to limited switching states of memristive devices. In this work, we propose the design of an analog deep neural network with binary weight update through backpropagation algorithm using binary state memristive devices. We show that such networks can be successfully used for image processing task and has the advantage of lower power consumption and small on-chip area in comparison with digital counterparts. The proposed network was benchmarked for MNIST handwritten digits recognition achieving an accuracy of approximately 90%.
We study the stability and information encoding capacity of synchronized states in a neuronal network model that represents part of thalamic circuitry. Our model neurons have a Hodgkin-Huxley-type low threshold Calcium channel, display post inhibitory rebound, and are connected via GABAergic inhibitory synapses. We find that there is a threshold in synaptic strength, $tau_c$, below which there are no stable spiking network states. Above threshold the stable spiking state is a cluster state, where different groups of neurons fire consecutively, and each neuron fires with the same cluster each time. Weak noise destabilizes this state, but stronger noise drives the system into a different, self-organized, stochastically synchronized state. Neuronal firing is still organized in clusters, but individual neurons can hop from cluster to cluster. Noise can actually induce and sustain such a state below the threshold of synaptic strength. We do find a qualitative difference in the firing patterns between small ($sim 10$ neurons) and large ($sim 1000$ neurons) networks. We determine the information content of the spike trains in terms of two separate contributions: the spike time jitter around cluster firing times, and the hopping from cluster to cluster. We quantify the information loss due to temporally correlated interspike intervals. Recent experiments on the locust olfactory system and striatal neurons suggest that the nervous system may actually use these two channels to encode separate and unique information.
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
