Three-layered hierarchical scheme with a Kinect sensor microphone array for audio-based human behavior recognition

•Proposed three-layered hierarchical human behavior recognition uses only audio data.•The use of Kinect sensor microphone array for data capture and fusion is explored.•The proposed scheme increased recognition robustness compared with conventional GMM.

This study develops a hierarchical scheme with three processing layers for human behavior recognition. The proposed scheme is an audio-based approach that employs a microphone array of the Kinect sensor for sensing and acquiring acoustic data to classify human behavior. The three processing layers, namely the feature layer, acoustic event classification layer, and specific behavior recognition layer, are interrelated, and the sensing data fusion, Gaussian mixture model with a classification tree, and state machine diagram to regulate human behavior are employed in these three layers, respectively. With enhanced performance of the feature and acoustic event classification layers, the proposed scheme exhibits increased human behavior classification accuracy. Human behavior recognition experiments were conducted in a research office, and three specific office behavior modes, namely “Laboratory meeting,” “Classmate chatting,” and “Laboratory study interaction,” were effectively classified using the proposed method.

Graphical abstractFigure optionsDownload full-size imageDownload as PowerPoint slide