Uniform query formalization in mobile visual search: From standards to practice

Mobile visual search is the task of finding information (visual or not) given an input picture or video usually taken by a hand-held device. Interest around this topic is being boosted by the increasing amount of digitally stored images and the widespread proliferation of camera enabled mobile devices, such as mobile phones, PDAs or tablets. This fact is increasing the urgency of novel solutions for challenging problems such as the efficient coding of compact visual descriptors and the interoperability of distributed visual search query interfaces. Currently, almost every visual search service offers a different retrieval interface and image metadata description format, preventing unified and efficient access. In this context, standardization groups such as ISO/IEC SC29/WG11 (MPEG) and ISO/IEC SC29/WG1 (JPEG) have been working to create unified interfaces for image repositories. In one hand, MPEG provides the ISO/IEC 15 938-12 (MPEG Query Format, MPQF), which standardizes a query language for multimedia repositories and has also started an activity for standardizing compact descriptors for visual search (CDVS). On the other hand, JPEG is now finishing the ISO/IEC 24800 (JPSearch), which provides solutions to the image metadata interoperability problem. This paper analyzes how these standardization activities can be combined to satisfy the requirements posed by the mobile visual search scenario, which are their limitations and which would be the necessary actions to be taken by the standardization committees in order to overcome them.