A generative framework for real time object detection and classification

We formulate a probabilistic model of image generation and derive optimal inference algorithms for finding objects and object features within this framework. The approach models images as a collage of patches of arbitrary size, some of which contain the object of interest and some of which are background. The approach requires development of likelihood-ratio models for object versus background generated patches. These models are learned using boosting methods. One advantage of the generative approach proposed here is that it makes explicit the conditions under which it is optimal. We applied the approach to the problem of finding faces and eyes on arbitrary images. Optimal inference under the proposed model works in real time and is robust to changes in lighting, illumination, and differences in facial structure, including facial expressions and eyeglasses. Furthermore, the system can simultaneously track the eyes and blinks of multiple individuals. Finally we reflect on how the development of perceptive systems like this may help advance our understanding of the human brain.