An energy efficient Reinforcement Learning based Cooperative Channel Sensing for Cognitive Radio Sensor Networks

In Cognitive Radio (CR), the conventional narrow band spectrum sensing requires either random channel sensing order or predefined channel sensing sequence to sense all channels in a specified spectrum band in order to detect vacant channels. This may be inefficient in energy constraint devices networks such as Cognitive Radio Wireless Sensor Network (CR-WSN). In this paper, we propose a Reinforcement Learning based clustered Cooperative Channel Sensing (RL-CCS) that learns channels' dynamic behaviors in terms of channel availability, sensing energy cost, and channel impairment to achieve optimal sensing sequence and optimal set of channels. The problem of selecting optimal policy is formulated as a Markov Decision Problem (MDP) to determine optimal solutions that minimize sensing energy while improving Primary User (PU) detection and channel utilization in CR-WSN. Simulation results show convergence and adaptability of the algorithm to dynamic environment in achieving optimal solutions. The results also indicate that with optimal channel sensing sequence and optimal sets of channels, sensing energy cost savings of 15.14% per channel sensing cycle can be achieved while improving PU detection accuracy and channel utilization compared to the sensing sequence based on Greedy search approach. Performance comparison of the proposed algorithm with other benchmark schemes indicates viability of our proposed approach over the other schemes in minimizing sensing energy and improving PU detection performance.