Adaptive guided image filter for warping in variational optical flow computation

•Introducing an adaptive guided image filter to correct errors of the intermediate warped interpolation image.•Reconstructing a guidance image as a combination of the reference image and the warped image.•The regularization parameter is adaptively selected based on a confidence measure to adjust the smoothing degree.

The variational optical flow method is considered to be the standard method to calculate an accurate dense motion field between successive frames. It assumes that the energy function has spatiotemporal continuities and appearance motions are small. However, for real image sequences, the temporal continuity assumption is often violated due to outliers and occlusions, causing inaccurate flow vectors at these regions. After each warping operation, errors are generated at the corresponding regions of the warped interpolation image. This results in an inaccurate discrete approximation of the temporal derivative and thus ends up affecting the accuracy of the estimated flow field. In this paper, we propose an adaptive guided image filter to correct these errors in the warped interpolation image. A guidance image is reconstructed by considering both the feature of the reference image as well as the difference between the warped interpolation image and the reference image, to guide the filtering of the warped interpolation image. To adjust the smoothing degree, the regularization parameter in the guided image filter is adaptively selected based on a confidence measure. Extensive experiments on different datasets and comparison with state-of-the-art variational optical flow algorithms demonstrate the effectiveness of our method.