Reinforcement Learning Spectrum Management Paradigm in Cognitive Radio using Novel State and Action Sets

This paper proposes a reinforcement learning(RL) model for cognitive radio(CR). By using this model, cognitive base station(CBS) can preform two-step decision of channel allocation, that is, whether to switch the channel for CR users and how to select the best channel if the CBS decides to switch, to avoid excessive channel switch and improve the throughput of the unlicensed user. Also, the performance of RL spectrum management depends highly on exploration strategy. Epsilon-greedy exploration method is used to solve the balance problem of RL decision process. Simulation results show that the implementation of the epsilon-greedy in each decision step has a remarkable effect on the system performance. The proposed method is superior to traditional RL spectrum allocation scheme in improving unlicensed users' throughput and reducing channel switch.