High quality, low delay foveated visual communications over mobile channels

Foveated video streams are created by selectively varying the spatio-temporal resolution of video data according to the assumed or measured fixation patterns of the intended viewers. The significant potential of foveated video lies in the considerable entropy reduction relative to the original video data while minimizing the loss of apparent visual information. By exploiting new human-machine interface techniques, such as visual eyetrackers, combined with foveated video compression and communication protocols, more efficient visual multimedia services operating over low bandwidths should become available in the near future. In this paper, we introduce a prototype foveated visual communication system suitable for implementation as a core element of an interactive multimedia wireless communication environment. We demonstrate the benefit of using foveation in noisy wireless low bandwidth applications, using measured fading statistics from the downtown area of Austin, Texas as an example. Based on a maximum source throughput criterion, the source and channel video transmission rates and the target video coding bit rate are adaptively decided according to the channel dynamics. In the simulations, we use the channel throughput, the spatial/temporal resolution, and the transmission delay as criteria to compare the performance of the foveated approach relative to normal (non-foveated) video. The results clearly underline the significant potential of foveated video communication protocols for wireless multimedia applications.