Evaluation of two stereo matchers on long real-world video sequences

• We evaluate iterative semi-global matching and linear belief-propagation matching.
• The evaluation is on long real-world video sequences where disparity ground-truth is not available.
• We apply the third-eye method and new data measures on video sequences.
• We propose how to compare evaluation measures relatively to each other.
• We suggest an evaluation based on data measures only.

The paper evaluates iterative semi-global matching (iSGM) and linear belief-propagation matching (linBPM), both using a census data-cost function, which are two of the currently top-ranked stereo matchers. The evaluation is on long real-world video sequences where disparity ground-truth is not available. The paper applies two alternative (or mutually supporting) techniques for performance evaluation: the previously known third-eye method, and a few new data measures on video sequences. The main contribution of the paper is on answering the questions, how to evaluate stereo matchers on long real-world sequences if disparity ground truth is not available, and how to compare evaluation measures relatively to each other. The two stereo matchers used are illustrating the discussed evaluation measures; they could be replaced by other matchers, but evaluation results for those two matchers are also of interest on its own, by illustrating correlations in the behavior of those two basically very different matchers (defined by dynamic programming or by belief propagation optimization, respectively) on data sequences recorded in different traffic situations.