Temporal stochastic linear encoding networks

Convolutional Neural Networks (CNNs) have achieved great success for action recognition. Technically, extracting effective long-range temporal dynamics is critical for such temporal tasks. This paper proposes a temporal stochastic linear encoding (TSLE) to construct the global video representations for action recognition, which can be embedded inside of CNN as a layer. The advantages of temporal stochastic linear encoding networks (TSLEN) are: (a) Compared with algorithms focusing on the short-term motions it can implement an easy yet robust manipulation of long range temporal clues. (b) We propose an arbitrarily directional motion boundary (ADMB) unit, which can save the training time and hard disk space. (c) The proposed TSLE unit maps the highly-dimensional videos to the compact spatio-temporal representations. On the efficiency and recognition accuracy experimental results demonstrate that the proposed TSLENs achieve competitive performance among the effective algorithms.