An ultra-fast human detection method for color-depth camera

•An extremely fast technique to locate positions that are plausibly humans is proposed to quickly reduce searching space.•A novel knowledge based human locator which can deal with partial occlusion and incomplete depth data is proposed.•A stepwise filtering framework enables the system to perform very quickly (80–140 fps).

Real-time human detection is important for a wide range of applications. The task is highly challenging due to occlusions, complex backgrounds, and variation of human poses. We propose a cascade-structured approach to real-time human detection in cluttered and dynamic environments with both color and depth data seamlessly incorporated. The first stage efficiently exploits depth data which generates a set of physically plausible yet over-detected candidates. These candidates are then purified by another two filters: a knowledge based human upper portion locator and a data-driven learning based filter. Experimental results show high detection accuracy achieved by the proposed method at 80–140 fps on a single CPU core (without GPU acceleration).