The Q-learning obstacle avoidance algorithm based on EKF-SLAM for NAO autonomous walking under unknown environments

The two important problems of SLAM and Path planning are often addressed independently. However, both are essential to achieve successfully autonomous navigation. In this paper, we aim to integrate the two attributes for application on a humanoid robot. The SLAM problem is solved with the EKF-SLAM algorithm whereas the path planning problem is tackled via Q-learning. The proposed algorithm is implemented on a NAO equipped with a laser head. In order to differentiate different landmarks at one observation, we applied clustering algorithm on laser sensor data. A Fractional Order PI controller (FOPI) is also designed to minimize the motion deviation inherent in during NAO's walking behavior. The algorithm is tested in an indoor environment to assess its performance. We suggest that the new design can be reliably used for autonomous walking in an unknown environment.