Supervised Neural Q_learning based Motion Control for Bionic Underwater Robots

Bionic underwater robots have been a hot research area in recent years. The motion control methods for a kind of bionic underwater robot with two undulating fins are discussed in this paper. The equations of motion for the bionic underwater robot are described. To apply the reinforcement learning to the actual robot control, a Supervised Neural Q_learning (SNQL) algorithm is put forward. This algorithm is based on conventional Q_learning algorithm, but has three remarkable distinctions: (1) using a feedforward neural network to approximate the Q_function table; (2) adopting a learning sample database to speed up learning and improve the stability of learning system; (3) introducing a supervised control in the earlier stage of learning for safety and to speed up learning again. Experiments of swimming straightforward are carried out with SNQL algorithm. Results indicate that the SNQL algorithm is more effective than pure neural Q_learning or supervised control. It is a feasible approach to figure out the motion control for bionic underwater robots.