Multiclass object classification for real-time video surveillance systems

Object classification in video is an important factor for improving the reliability of various automatic applications in video surveillance systems, as well as a fundamental feature for advanced applications, such as scene understanding. Despite extensive research, existing methods exhibit relatively moderate classification accuracy when tested on a large variety of real-world scenarios, or do not obey the real-time constraints of video surveillance systems. Moreover, their performance is further degraded in multi-class classification problems. We explore multi-class object classification for real-time video surveillance systems and propose an approach for classifying objects in both low and high resolution images (human height varies from a few to tens of pixels) in varied real-world scenarios. Firstly, we present several features that jointly leverage the distinction between various classes. Secondly, we provide a feature-selection procedure based on entropy gain, which screens out superfluous features. Experiments, using various classification techniques, were performed on a large and varied database consisting of ∼29,000 object instances extracted from 140 different real-world indoor and outdoor, near-field and far-field scenes having various camera viewpoints, which capture a large variety of object appearances under real-world environmental conditions. The insight raised from the experiments is threefold: the efficiency of our feature set in discriminating between classes, the performance improvement when using the feature selection method, and the high classification accuracy obtained on our real-time system on both DSP (TMS320C6415-6E3, 600 MHz) and PC (Quad Core Intel® Xeon® E5310, 2 × 4 MB Cache, 1.60 GHz, 1066 MHz) platforms.

Research highlights► We present a real-time object classifier for video surveillance systems. ► We use a large database extracted from various real-world scenes and scenarios. ► We present features that jointly leverage the distinction between various classes. ► We show the performance improvement when using our feature selection method. ► We demonstrate high classification accuracy on real-time systems.