An entropy-based evaluation method for knowledge bases of medical information systems

• Development of an entropy-based evaluation method to evaluate ontology strength.
• Evaluation an ontological semantic ontology using the evaluation method.
• Evaluation of the backbone of the UMLS with this method.

In this paper we introduce a method to develop knowledge bases for medical decision support systems, with a focus on evaluating such knowledge bases. Departing from earlier efforts with concept maps, we developed an ontological-semantic knowledge base and evaluated its information content using the metrics we have developed, and then compared the results to the UMLS backbone knowledge base. The evaluation method developed uses information entropy of concepts, but in contrast to previous approaches normalizes it against the number of relations to evaluate the information density of knowledge bases of varying sizes. A detailed description of the knowledge base development and evaluation is discussed using the underlying algorithms, and the results of experimentation of the methods are explained. The main evaluation results show that the normalized metric provides a balanced method for assessment and that our knowledge base is strong, despite having fewer relationships, is more information-dense, and hence more useful. The key contributions in the area of developing expert systems detailed in this paper include: (a) introduction of a normalized entropy-based evaluation technique to evaluate knowledge bases using graph theory, (b) results of the experimentation of the use of this technique on existing knowledge bases.