New challenges in wireless and free space optical communications

•A survey on wireless communication is presented.•RF recent challenges and research studies are described.•FSO up to date issues are briefly described.•Our various contributions are summarized.

This manuscript presents a survey on new challenges in wireless communication systems and discusses recent approaches to address some recently raised problems by the wireless community. At first a historical background is briefly introduced. Challenges based on modern and real life applications are then described. Up to date research fields to solve limitations of existing systems and emerging new technologies are discussed. Theoretical and experimental results based on several research projects or studies are briefly provided. Essential, basic and many self references are cited. Future researcher axes are briefly introduced.