Visible models for interactive pattern recognition

The exchange of information between human and machine has been a bottleneck in interactive visual classification. The visible model of an object to be recognized is an abstraction of the object superimposed on its picture. It is constructed by the machine but it can be modified by the operator. The model guides the extraction of features from the picture. The classes are rank ordered according to the similarities (in the hidden high-dimensional feature space) between the unknown picture and a set of labeled reference pictures. The operator can either accept one of the top three candidates by clicking on a displayed reference picture, or modify the model. Model adjustment results in the extraction of new features, and a new rank ordering. The model and feature extraction parameters are re-estimated after each classified object, with its model and label, is added to the reference database. Pilot experiments show that interactive recognition of flowers and faces is more accurate than automated classification, faster than unaided human classification, and that both machine and human performance improve with use.