Training Convolutional Neural Networks (CNNs) usually requires a large number of computational resources. In this paper, textit{SparseTrain} is proposed to accelerate CNN training by fully exploiting the sparsity. It mainly involves three levels of i
nnovations: activation gradients pruning algorithm, sparse training dataflow, and accelerator architecture. By applying a stochastic pruning algorithm on each layer, the sparsity of back-propagation gradients can be increased dramatically without degrading training accuracy and convergence rate. Moreover, to utilize both textit{natural sparsity} (resulted from ReLU or Pooling layers) and textit{artificial sparsity} (brought by pruning algorithm), a sparse-aware architecture is proposed for training acceleration. This architecture supports forward and back-propagation of CNN by adopting 1-Dimensional convolution dataflow. We have built %a simple compiler to map CNNs topology onto textit{SparseTrain}, and a cycle-accurate architecture simulator to evaluate the performance and efficiency based on the synthesized design with $14nm$ FinFET technologies. Evaluation results on AlexNet/ResNet show that textit{SparseTrain} could achieve about $2.7 times$ speedup and $2.2 times$ energy efficiency improvement on average compared with the original training process.
Sparsification is an efficient approach to accelerate CNN inference, but it is challenging to take advantage of sparsity in training procedure because the involved gradients are dynamically changed. Actually, an important observation shows that most
of the activation gradients in back-propagation are very close to zero and only have a tiny impact on weight-updating. Hence, we consider pruning these very small gradients randomly to accelerate CNN training according to the statistical distribution of activation gradients. Meanwhile, we theoretically analyze the impact of pruning algorithm on the convergence. The proposed approach is evaluated on AlexNet and ResNet-{18, 34, 50, 101, 152} with CIFAR-{10, 100} and ImageNet datasets. Experimental results show that our training approach could substantially achieve up to $5.92 times$ speedups at back-propagation stage with negligible accuracy loss.
Bayesian inference is an effective approach for solving statistical learning problems, especially with uncertainty and incompleteness. However, Bayesian inference is a computing-intensive task whose efficiency is physically limited by the bottlenecks
of conventional computing platforms. In this work, a spintronics based stochastic computing approach is proposed for efficient Bayesian inference. The inherent stochastic switching behaviors of spintronic devices are exploited to build stochastic bitstream generator (SBG) for stochastic computing with hybrid CMOS/MTJ circuits design. Aiming to improve the inference efficiency, an SBG sharing strategy is leveraged to reduce the required SBG array scale by integrating a switch network between SBG array and stochastic computing logic. A device-to-architecture level framework is proposed to evaluate the performance of spintronics based Bayesian inference system (SPINBIS). Experimental results on data fusion applications have shown that SPINBIS could improve the energy efficiency about 12X than MTJ-based approach with 45% design area overhead and about 26X than FPGA-based approach.
We report an inelastic neutron scattering study of the spin fluctuations in the nearly-ferromagnetic element palladium. Dispersive over-damped collective magnetic excitations or ``paramagnons are observed up to 128 meV. We analyze our results in term
s of a Moriya-Lonzarich-type spin fluctuation model and estimate the contribution of the spin fluctuations to the low temperature heat capacity. In spite of the paramagnon excitations being relatively strong, their relaxation rates are large. This leads to a small contribution to the low-temperature electronic specific heat.
We review neutron scattering investigations of the crystal structures, magnetic structures, and spin dynamics of the iron-based RFe(As,P)O (R=La, Ce, Pr, Nd), (Ba,Sr,Ca)Fe2As2, and Fe1+x(Te-Se) systems. On cooling from room temperature all the undope
d materials exhibit universal behavior, where a tetragonal-to-orthorhombic/monoclinic structural transition occurs, below which the systems become antiferromagnets. For the first two classes of materials the magnetic structure within the a-b plane consists of chains of parallel Fe spins that are coupled antiferromagnetically in the orthogonal direction, with an ordered moment typically less than one Bohr magneton. Hence these are itinerant electron magnets, with a spin structure that is consistent with Fermi-surface nesting and a very energetic spin wave bandwidth ~0.2 eV. With doping, the structural and magnetic transitions are suppressed in favor of superconductivity. Magnetic correlations are observed in the superconducting regime, with a magnetic resonance that follows the superconducting order parameter just like the cuprates. The rare-earth moments order antiferromagnetically at low T like conventional magnetic-superconductors. Pressure in CaFe2As2 transforms the system from a magnetically ordered orthorhombic material to a collapsed non-magnetic tetragonal system. Tetragonal Fe1+xTe transforms to a low T monoclinic structure at small x that changes to orthorhombic at larger x, which is accompanied by a crossover from commensurate to incommensurate magnetic order. Se doping suppresses the magnetic order.
Neutron scattering has been used to investigate the evolution of the long- and short-range charge-ordered (CO), ferromagnetic (FM), and antiferromagnetic (AF) correlations in single crystals of Pr1-xCaxMnO3. The existence and population of spin clust
ers as refected by short-range correlations are found to drastically depend on the doping (x) and temperature (T). Concentrated spin clusters coexist with long-range canted AF order in a wide temperature range in x = 0.3 while clusters do not appear in x = 0.4 crystal. In contrast, both CO and AF order parameters in the x = 0.35 crystal show a precipitous decrease below ~ 35 K where spin clusters form. These results provide direct evidence of magnetic phase separation and indicate that there is a critical doping x_c (close to x = 0.35) that divides the phase-separated site-centered from the homogeneous bond-centered or charge-disproportionated CO ground state.
We use neutron scattering to study the Pr$^{3+}$ crystalline electric field (CEF) excitations in the filled skutterudite PrOs$_4$As$_{12}$. By comparing the observed levels and their strengths under neutron excitation with the theoretical spectrum an
d neutron excitation intensities, we identify the Pr$^{3+}$ CEF levels, and show that the ground state is a magnetic $Gamma_4^{(2)}$ triplet, and the excited states $Gamma_1$, $Gamma_4^{(1)}$ and $Gamma_{23}$ are at 0.4, 13 and 23 meV, respectively. A comparison of the observed CEF levels in PrOs$_4$As$_{12}$ with the heavy fermion superconductor PrOs$_4$Sb$_{12}$ reveals the microscopic origin of the differences in the ground states of these two filled skutterudites.
We use neutron scattering to study the lattice and magnetic structure of the layered half-doped manganite Pr$_{0.5}$Ca$_{1.5}$MnO$_4$. On cooling from high temperature, the system first becomes charge- and orbital- ordered (CO/OO) near $T_{CO}=300$ K
and then develops checkerboard-like antiferromagnetic (AF) order below $T_{N}=130$ K. At temperatures above $T_{N}$ but below $T_{CO}$ ($T_N<T<T_{CO}$), the appearance of short-range AF spin correlations suppresses the CO/OO induced orthorhombic strain, contrasting with other half-doped manganites, where AF order has no observable effect on the lattice distortion. These results suggest that a strong spin-lattice coupling and the competition between AF exchange and CO/OO ordering ultimately determines the low-temperature properties of the system.
We use inelastic neutron scattering to probe magnetic excitations of an optimally electron-doped superconductor Nd$_{1.85}$Ce$_{0.15}$CuO$_{4-delta}$ above and below its superconducting transition temperature $T_c=25$ K. In addition to gradually open
ing a spin pseudo gap at the antiferromagnetic ordering wavevector ${bf Q}=(1/2,1/2,0)$, the effect of superconductivity is to form a resonance centered also at ${bf Q}=(1/2,1/2,0)$ but at energies above the spin pseudo gap. The intensity of the resonance develops like a superconducting order parameter, similar to those for hole-doped superconductors and electron-doped Pr$_{0.88}$LaCe$_{0.12}$CuO$_4$. The resonance is therefore a general phenomenon of cuprate superconductors, and must be fundamental to the mechanism of high-$T_c$ superconductivity.
We use inelastic neutron scattering to explore the evolution of the low energy spin dynamics in the electron-doped cuprate Pr0.88LaCe0.12CuO4-d (PLCCO) as the system is tuned from its nonsuperconducting, as-grown antiferromagnetic (AF) state into an
optimally-doped superconductor (Tc~24 K) without static AF order. The low temperature, low energy response of the spin excitations in under-doped samples is coupled to the presence of the AF phase, whereas the low-energy magnetic response for samples near optimal Tc exhibits spin fluctuations surprisingly insensitive to the sample temperature. This evolution of the low energy excitations is consistent with the influence of a quantum critical point in the phase diagram of PLCCO associated with the suppression of the static AF order. We carried out scaling analysis of the data and discuss the influence of quantum critical dynamics in the observed excitation spectrum.
