An end-to-end virtual environment streaming technique for thin mobile devices over heterogeneous networks

Advances in mobile computing devices have resulted in a new class of 3D virtual environment applications such as virtual guides and malls, online gaming, training and monitoring systems, just to name a few. The challenge lies in determining how to provide a rich and detailed 3D virtual environment to these thin devices, since they are known for their lack of the proper resources required to process large scale 3D geometric data. In a wireless scenario, in which bandwidth is highly dynamic, users may experience long delays when downloading an entire 3D environment before they start interacting with it. Furthermore, thin wireless mobile devices are unable to handle the complexity of 3D graphics at interactive frame rates. In this research work, we propose efficient end-to-end streaming and rate control protocols. Our approach consists of moving the demanding geometry-rendering task to a dedicated remote rendering server that streams the rendering output to a client, leaving only the displaying and certain minor image-based rendering tasks to the local, less powerful mobile hardware. We also investigate an image buffer management mechanism in order to deal with the small storage available on thin devices. We discuss the design of our proposed solutions and report on their performance evaluation through an extensive set of simulation experiments.