Multimodal medical information retrieval with unsupervised rank fusion

• We propose a medical retrieval system supporting text and image queries.
• It can retrieve either relevant PubMed articles or images from those articles.
• It supports automatic query term expansion from the MeSH thesaurus.
• Novel fusion algorithm, ISR, improves results from existing rank fusion algorithms.
• We got the best result on multimodal case-based retrieval in 2013 ImageCLEFMedical

Modern medical information retrieval systems are paramount to manage the insurmountable quantities of clinical data. These systems empower health care experts in the diagnosis of patients and play an important role in the clinical decision process. However, the ever-growing heterogeneous information generated in medical environments poses several challenges for retrieval systems. We propose a medical information retrieval system with support for multimodal medical case-based retrieval. The system supports medical information discovery by providing multimodal search, through a novel data fusion algorithm, and term suggestions from a medical thesaurus. Our search system compared favorably to other systems in 2013 ImageCLEFMedical.