To regularize or not: Revisiting SGD with simple algorithms and experimental studies

This study provides insights on using SGD algorithms for big data applications, e.g., to accelerate SVMs or regularized regression in large-scale, or to improve the performance of online learning or real time forecasting (control). In particular, when T is pre-specified, SGDE (without an explicit regularizer) can give us a well enough perforce with simplicity and understandability; when T cannot be pre-specified, ASG can improve the performance of standard SGD algorithms.