Accurate abandoned and removed object classification using hierarchical finite state machine

• We propose a novel and accurate ARO classification method.
• We propose a hierarchical FSM consisting of pixel-, region-, and event-layers.
• State transition is done by the pre-trained SVM using 7 different input features.
• The proposed ARO method shows higher classification and low false alarm.
• The proposed ARO method can be applied to many practical applications.

The ability of most existing approaches to classify abandoned and removed objects (AROs) in images is affected by external environmental conditions such as illumination and traffic volume because the approaches use several pre-defined threshold values and generate many falsely-classified static regions. To reduce these effects, we propose an accurate ARO classification method using a hierarchical finite state machine (FSM) that consists of pixel-layer, region-layer, and event-layer FSMs, where the result of the lower-layer FSM is used as the input of the higher-layer FSM. Each FSM is defined by a Mealy state machine with three states and several state transitions, where a support vector machine (SVM) determines the state transition based on the current state and input features such as area, intensity, motion, shape, time duration, color and edge. Because it uses the hierarchical FSM (H-FSM) structure with features that are optimally trained by SVM classifiers, the proposed ARO classification method does not require threshold values and guarantees better classification accuracy under severe environmental changes. In experiments, the proposed ARO classification method provided much higher classification accuracy and lower false alarm rate than the state-of-the-art methods in both public databases and a commercial database. The proposed ARO classification method can be applied to many practical applications such as detection of littering, illegal parking, theft, and camouflaged soldiers.