Illumination invariant optical flow using neighborhood descriptors

•Optical flow approach robust towards illumination changes and texture variability.•Accurate multiscale TV-l1 approach for both small and large displacements.•Accurate results for very different scenes with constant algorithm parameters.•High performance was obtained on the Middlebury, KITTI and MPI Sintel databases.•The algorithm enabled mosaicing of endoscopic images under different modalities.

Total variational (TV) methods using l1-norm are efficient approaches for optical flow determination. This contribution presents a multi-resolution TV-l1 approach using a data-term based on neighborhood descriptors and a weighted non-local regularizer. The proposed algorithm is robust to illumination changes. The benchmarking of the proposed algorithm is done with three reference databases (Middlebury, KITTI and MPI Sintel). On these databases, the proposed approach exhibits an optimal compromise between robustness, accuracy and computation speed. Numerous tests performed both on complicated data of the reference databases and on challenging endoscopic images acquired under three different modalities demonstrate the robustness and accuracy of the method against the presence of large or small displacements, weak texture information, varying illumination conditions and modality changes.