Good recognition is non-metric

Author-Highlights
• A thorough and critical review of the most recent literature in metric learning and related fields.
• A new general definition of recognition that is not restricted to pair matching.
• An extensive meta-analysis of metric learning on Caltech 101 and Labeled Faces in the Wild.
• An experimental evaluation of top performing metric and non-metric algorithms.
• A series of useful recommendations, based on our results, for recognition algorithm designs.

Recognition is the fundamental task of visual cognition, yet how to formalize the general recognition problem for computer vision remains an open issue. The problem is sometimes reduced to the simplest case of recognizing matching pairs, often structured to allow for metric constraints. However, visual recognition is broader than just pair-matching: what we learn and how we learn it has important implications for effective algorithms. In this review paper, we reconsider the assumption of recognition as a pair-matching test, and introduce a new formal definition that captures the broader context of the problem. Through a meta-analysis and an experimental assessment of the top algorithms on popular data sets, we gain a sense of how often metric properties are violated by recognition algorithms. By studying these violations, useful insights come to light: we make the case for local distances and systems that leverage outside information to solve the general recognition problem.