Spatiotemporal bag-of-features for early wildfire smoke detection

•We select key frames from a video and detect candidate blocks only in key frames.•We prepare 3D spatiotemporal volumes by combining the candidate blocks.•We introduce a new weighting scheme for generating a more reasonable BoF.•The random forest classifier is built during the training phase by using the BoF.

Wildfire smoke detection is particularly important for early warning systems, because smoke usually rises before flames arise. Therefore, this paper presents an automatic wildfire smoke detection method using computer vision and pattern recognition techniques. First, candidate blocks are identified using key-frame differences and nonparametric smoke color models to detect smoke-colored moving objects. Subsequently, three-dimensional spatiotemporal volumes are built by combining the candidate blocks in the current key-frame with the corresponding blocks in previous frames. A histogram of oriented gradient (HOG) is extracted, and a histogram of oriented optical flow (HOOF) is extracted as a temporal feature based on the fact that the direction of smoke diffusion is upward owing to thermal convection. From spatiotemporal features of training data, a visual codebook and a bag-of-features (BoF) histogram are generated using our proposed weighting scheme. For smoke verification, a random forest classifier is built during the training phase using the BoF histogram. The random forest with the BoF histogram can increase the detection accuracy performance when compared with related methods and allow smoke detection to be carried out in near real time.