Rate-coded DBN: An online strategy for spike-based deep belief networks

The memory wall or Von Neumann memory bottleneck decreases the speed of computation in conventional digital platforms. It is due to disharmony of communication speed as well as physical distance between the CPU and memory. In Spiking Neural Network (SNN), both memory and computational elements are integrated into the body of each neuron which provides the possibility of cognitive computing with learning ability in a platform without memory bottleneck. The way of updating the synaptic weights is one of the significant challenges in implementing machine learning methods using spiking model of neurons. Deep Belief learning method recently is used in several state-of-the-art studies due to its potential in robustness computation, pattern recognition, and data classification. In this paper, we use a rate based version of Contrastive Divergence (CD) updating weight rule. Respecting the rate aspect of neural coding, we develop a Spike-Based Deep Belief Network (S-DBN) using Leaky Integrate-and-Fire (LIF) neurons. In experimental evaluations, a two stacked RBMs model is validated using MNIST hand-written digit dataset. The learning accuracy of this architecture is 94.9% yet less than state-of-the-art of Artificial Neural Network (ANN) accuracy, however, quite promising with SNN architecture suitable for hardware implementation. To enhance the accuracy of the network recognition rate, we studied the impacts of notable network parameters for both learning as well as neuron models. Based on the experimental results in this work, choosing optimized parameters for learning rate between 0.008 and 0.01, a frequency of input spikes 18 Hz, the sizes of mini-batches between 25 and 50, and two different membrane voltage thresholds for hidden and visible layers leads to the optimum results in the accuracy rate of the network recognition.