Online learning for vector autoregressive moving-average time series prediction

Multivariate time series analysis considers simultaneously multiple time series, which is much more complicated than the univariate time series analysis in general. VARMA (vector autoregressive moving-average) is one of the most mainstream multivariate time series model for time series prediction. However, the parameters of VARMA are often estimated in a batch manner in traditional multivariate statistical analysis and the noise terms are assumed Gaussian. In addition, the batch methods cannot perform satisfactorily in real-time prediction and the noise terms are unknown to us in real world. In this paper, we propose a novel online time series prediction framework for VARMA. We prove that VAR (vector autoregressive) can be used to mimic the underlying VARMA model under the online settings. Under this framework, we develop two effective algorithms VARMA-OGD and VARMA-ONS for this prediction problem assuming that the noise terms are generated stochastically and independently. The VARMA-OGD algorithm is based on the OGD (online gradient descent) algorithm, which is valid for general convex loss function. While the VARMA-ONS algorithm adopting the ONS (online newton step) algorithm is only valid for exp-concave loss function. Theoretical analysis shows that the regret bounds against the best VARMA prediction in hindsight of the proposed algorithms are sublinear of the number of samples. Furthermore, our experimental results further validate the effectiveness and robustness of our algorithms.