In this work, we are concerned with a Fokker-Planck equation related to the nonlinear noisy leaky integrate-and-fire model for biological neural networks which are structured by the synaptic weights and equipped with the Hebbian learning rule. The equation contains a small parameter $varepsilon$ separating the time scales of learning and reacting behavior of the neural system, and an asymptotic limit model can be derived by letting $varepsilonto 0$, where the microscopic quasi-static states and the macroscopic evolution equation are coupled through the total firing rate. To handle the endowed flux-shift structure and the multi-scale dynamics in a unified framework, we propose a numerical scheme for this equation that is mass conservative, unconditionally positivity preserving, and asymptotic preserving. We provide extensive numerical tests to verify the schemes properties and carry out a set of numerical experiments to investigate the models learning ability, and explore the solutions behavior when the neural network is excitatory.