Influence zones: A strategy to enhance reinforcement learning

Reinforcement Learning (RL) aims to learn through direct experimentation how to solve decision-making problems. RL algorithms often have their practical applications restricted to small or medium size problems—mainly because of their strategies for value function estimation demanding very high number of interactions. To overcome this difficulty, we propose to enhance RL performance by updating several state (or state–action) values at each interaction. Therefore, the influence zone algorithm, an improvement over the topological RL agent (TRLA) strategy, allows to reduce the number of requested interactions. Such a reduction is based on the topological-preserving characteristic of the mapping between states (or state–action pairs) and value estimates. The comparison of the influence zone approach with seven other RL algorithms suggests that the proposed algorithm is among the fastest to estimate the value function and that it takes less value function updatings to perform such an estimation. The influence zone algorithm also presents a remarkable flexibility in adapting its policy to changes of the input space topology.