Integrated detection and tracking of workforce and equipment from construction jobsite videos

•To design a method to integrate the visual detection and tracking techniques•To achieve high precision and recall for detecting and tracking construction objects•To simultaneously detect and track multiple construction objects of interest

High definition (HD) video cameras have been used to record daily activities at construction jobsites into videos. These videos contain rich workforce and equipment information for site engineers and project managers to analyze construction productivity, monitor construction progress, inspect jobsite safety, etc. However, it is difficult to automatically retrieve such information from the videos, since existing methods for the detection of construction workforce and equipment could not reach high precision and recall at the same time. This paper presents a novel framework that integrates the visual tracking into the detection of construction workforce and equipment. The integration significantly improves the recall and meanwhile maintains high precision. The proposed framework has been tested in real construction jobsites. Although it does not process the jobsite videos in real time yet, the test results showed that the recall for the detection of construction workforce and equipment was improved by more than 30-50%, while maintain the precision at the same level.