Part based model and spatial–temporal reasoning to recognize hydraulic excavators in construction images and videos

Detection of earthmoving equipment in construction images and videos can increase the automation level of many construction management tasks such as productivity measurement, locating of machines, work-zone safety, and semantic image and video indexing. Some of the earthmoving plants, such as hydraulic excavator, have articulated shapes making them a difficult target for even state of the art object recognition algorithms. The goal of this paper is to develop a model for non-rigid equipment detection and pose estimation in construction images and videos. In this paper, we describe an object recognition system based on mixture of appearances of deformable body parts of the hydraulic excavator and compare its results with general Histogram of Oriented Gradient detectors in both images and videos. Then a spatial–temporal reasoning model is presented which uses time and space constraints of the excavators' moving patterns to improve the detection results in videos.

► A vision-based algorithm is developed to detect excavators in images and videos.
► General Histogram of Oriented Gradients and part-based approaches were evaluated.
► Histogram of Oriented Gradients was used as a classification feature.
► Spatial–temporal reasoning was employed to enhance detection rate in videos.