Computer vision and driver distraction: Developing a behaviour-flagging protocol for naturalistic driving data

• We developed and tested a computer vision solution for coding naturalistic driving videos.
• The epistemic and logistical limitations of current data coding protocols are discussed.
• Computer vision solution reduced data processing time by 55–75 percent compared to manual review of video.
• We achieved a 35–69 percent increase in target behaviour recognition and classification.

Naturalistic driving studies (NDS) allow researchers to discreetly observe everyday, real-world driving to better understand the risk factors that contribute to hazardous situations. In particular, NDS designs provide high ecological validity in the study of driver distraction. With increasing dataset sizes, current best practice of manually reviewing videos to classify the occurrence of driving behaviours, including those that are indicative of distraction, is becoming increasingly impractical. Current statistical solutions underutilise available data and create further epistemic problems. Similarly, technical solutions such as eye-tracking often require dedicated hardware that is not readily accessible or feasible to use. A computer vision solution based on open-source software was developed and tested to improve the accuracy and speed of processing NDS video data for the purpose of quantifying the occurrence of driver distraction. Using classifier cascades, manually-reviewed video data from a previously published NDS was reanalysed and used as a benchmark of current best practice for performance comparison. Two software coding systems were developed – one based on hierarchical clustering (HC), and one based on gender differences (MF). Compared to manual video coding, HC achieved 86 percent concordance, 55 percent reduction in processing time, and classified an additional 69 percent of target behaviour not previously identified through manual review. MF achieved 67 percent concordance, a 75 percent reduction in processing time, and classified an additional 35 percent of target behaviour not identified through manual review. The findings highlight the improvements in processing speed and correctly classifying target behaviours achievable through the use of custom developed computer vision solutions. Suggestions for improved system performance and wider implementation are discussed.