An adaptive control momentum method as an optimizer in the cloud

Many issues in the cloud can be transformed into optimization problems, where data is of high dimension and randomness. Thus, stochastic optimizing is a key to Autonomous Cloud. And one of the most significant discussions in this field is how to adapt the learning rate and convergent path dynamically. This paper proposes a gradient-based algorithm called Adacom, that is based on an adaptive control system and momentum. Critically inheriting the previous studies, a reference model is introduced to generate the update. The method reduces noise and decides on paths with less oscillation, while maintaining the accumulated learning rate. Due to system design properties, the method requires fewer hyper-parameters for tuning. We state the prospect of Adacom as a general optimizer in Autonomous Cloud, and explore the potential of Adacom for pervasive computing by the assumption of transition data. Then we demonstrate the convergence of Adacom theoretically. The evaluations over the simulated transition data prove the feasibility and superiority of Adacom with other gradient-based methods.