Automatic image annotation and semantic based image retrieval for medical domain

Automatic image annotation is the process of assigning meaningful words to an image taking into account its content. This process is of great interest as it allows indexing, retrieving, and understanding of large collections of image data. This paper presents a system used in the medical domain for three distinct tasks: image annotation, semantic based image retrieval and content based image retrieval. An original image segmentation algorithm based on a hexagonal structure was used to perform the segmentation of medical images. Image's regions are described using a vocabulary of blobs generated from image features using the K-means clustering algorithm. The annotation and semantic based retrieval task is evaluated for two annotation models: Cross Media Relevance Model and Continuous-space Relevance Model. Semantic based image retrieval is performed using the methods provided by the annotation models. The ontology used by the annotation process was created in an original manner starting from the information content provided by the Medical Subject Headings (MeSH). The experiments were made using a database containing color images retrieved from medical domain using an endoscope and related to digestive diseases.