Extending information unit across media streams for improving retrieval effectiveness

This paper specifically addresses the effectiveness of our theoretically-based two dimensional retrieval model for searching semantically synchronized media streams. Conventional IR systems, which support partial retrieval of synchronized media streams, retrieve “atomic units”, for example slides, pages and shots of underlying media streams such as presentations, electronic books, and lecture videos respectively. In contrast, our model is based upon the concept of an extended retrieval unit and thus retrieves dynamically integrated media streams comprising several atomic units both along and across media streams. In this paper, in addition to reviewing our model, we describe its system implementation that we developed for conducting experiments on several real-world datasets built from scratch. We then present extensive empirical results which demonstrate that our system outperforms (a) conventional systems based on atomic retrieval units, (b) single dimensional retrieval systems, which extend retrieval units over several atomic units of the same media and (c) cross-media retrieval systems, which extend atomic retrieval units across several media stream units. The results thus verify our claims regarding the effectiveness of our two dimensional retrieval model for retrieving meaningful units of synchronized media streams.