Adaptive learning rate of SpikeProp based on weight convergence analysis

A Spiking Neural Network (SNN) training using SpikeProp and its variants is usually affected by sudden rise in learning cost called surges. These surges cause diversion in the learning process and often cause it to fail as well. Researches have shown that proper learning rate is crucial to avoid these surges. In this paper, we perform weight convergence analysis to determine the proper step size in each iteration of weight update and derive an adaptive learning rate extension to SpikeProp that assures convergence of the learning process. We have analyzed the performance of this learning rate adaptation with existing methods via simulations on different benchmarks. The results show that using adaptive learning rate significantly improves the weight convergence and speeds up learning as well.