A graph-based approach for visualizing and exploring a multimedia search result space

• Innovative contribution to aggregated search.
• Innovative contribution to the definition of interfaces for visualizing search results.
• Graph-based visualization of multimedia search results.
• Multimodal access to media objects.
• User based evaluation.

Nowadays, together with the increasing spread of the content available online, users’ information needs have become more complex. To fulfill them, users strongly rely on Web search engines, but traditional ways of presenting search results are often unsatisfactory. In fact, Web pages carry information that exists in multiple media formats, such as text, audio, image and video objects. Vertical search engines and medium-specific search services do not provide users with an integrated view of search results. Furthermore, multiple media objects are in most of the cases highly semantically interlinked, but the connections between them are not sufficiently exploited to provide a further exploration of the retrieved objects. To address these issues, in this paper we propose a graph-based approach aimed at providing users with the possibility to dynamically visualize and explore a search result space built over a repository of multimedia documents containing interconnected multiple media objects. To do this, we represent the search result space via a graph-based data model, where both the retrieved multimedia documents and connected relevant media objects are considered. Media objects, among them, are connected via different kinds of similarity relationships, which depend on the low-level features and metadata taken into consideration to access the media objects. The approach and the connected visualization and exploration interface have been implemented and tested on a publicly available dataset, and they have been evaluated by means of a usability test.