Performance of distributed multi-agent multi-state reinforcement spectrum management using different exploration schemes

This paper introduces a novel multi-agent multi-state reinforcement learning exploration scheme for dynamic spectrum access and dynamic spectrum sharing in wireless communications. With the multi-agent multi-state reinforcement learning, cognitive radios can decide the best channels to use in order to maximize spectral efficiency in a distributed way. However, we argue that the performance of spectrum management, including both dynamic spectrum access and dynamic spectrum sharing, will largely depend on different reinforcement learning exploration schemes, and we believe that the traditional multi-agent multi-state reinforcement learning exploration schemes may not be adequate in the context of spectrum management. We then propose a novel reinforcement learning exploration scheme and show that we can improve the performance of multi-agent multi-state reinforcement learning based spectrum management by using the proposed reinforcement learning exploration scheme. We also investigate various real-world scenarios, and confirm the validity of the proposed method.

► We design decentralized cognitive radios for non-coordinated dynamic spectrum sharing.
► Multi-agent multi-state RL on DSS faces communication and learning interference.
► The Single-Agent-Learning-Oriented measure quantifies desirable behaviours.
► We propose a Enhanced Unequal Exploration (EUE) scheme for DSS problems.