RGB-D image-based detection of stairs, pedestrian crosswalks and traffic signs

• A new framework can be integrated into navigation and wayfinding aid for blind people.
• Effective methods to detect and recognize stairs and pedestrian crosswalks based on RGB-D images.
• A new method for recognizing traffic lights of pedestrian crosswalks.

A computer vision-based wayfinding and navigation aid can improve the mobility of blind and visually impaired people to travel independently. In this paper, we develop a new framework to detect and recognize stairs, pedestrian crosswalks, and traffic signals based on RGB-D (Red, Green, Blue, and Depth) images. Since both stairs and pedestrian crosswalks are featured by a group of parallel lines, we first apply Hough transform to extract the concurrent parallel lines based on the RGB (Red, Green, and Blue) channels. Then, the Depth channel is employed to recognize pedestrian crosswalks and stairs. The detected stairs are further identified as stairs going up (upstairs) and stairs going down (downstairs). The distance between the camera and stairs is also estimated for blind users. Furthermore, the traffic signs of pedestrian crosswalks are recognized. The detection and recognition results on our collected datasets demonstrate the effectiveness and efficiency of our proposed framework.