Neural inverse reinforcement learning in autonomous navigation

• A novel model-free algorithm, Neural Inverse Reinforcement Learning, is introduced in performing autonomous navigation tasks.
• A nonlinear neural policy representation is proposed to establish the mapping between the state and action spaces.
• Computer-based expert demonstrations are supplemented to learning robots when human experts are not available in some extreme navigation tasks.
• A refinement operation based on maximum a posteriori is designed to pretreat the demonstrations from suboptimal experts.
• This method can easily deal with large state spaces and generalize unvisited states in demonstrations.

Designing intelligent and robust autonomous navigation systems remains a great challenge in mobile robotics. Inverse reinforcement learning (IRL) offers an efficient learning technique from expert demonstrations to teach robots how to perform specific tasks without manually specifying the reward function. Most of existing IRL algorithms assume the expert policy to be optimal and deterministic, and are applied to experiments with relatively small-size state spaces. However, in autonomous navigation tasks, the state spaces are frequently large and demonstrations can hardly visit all the states. Meanwhile the expert policy may be non-optimal and stochastic. In this paper, we focus on IRL with large-scale and high-dimensional state spaces by introducing the neural network to generalize the expert’s behaviors to unvisited regions of the state space and an explicit policy representation is easily expressed by neural network, even for the stochastic expert policy. An efficient and convenient algorithm, Neural Inverse Reinforcement Learning (NIRL), is proposed. Experimental results on simulated autonomous navigation tasks show that a mobile robot using our approach can successfully navigate to the target position without colliding with unpredicted obstacles, largely reduce the learning time, and has a good generalization performance on undemonstrated states. Hence prove the robot intelligence of autonomous navigation transplanted from limited demonstrations to completely unknown tasks.