Automatic verification of a knowledge base by using a multi-criteria group evaluation with application to security screening at an airport

• A new method of automatic verification of knowledge base is presented.
• The method uses a multi-criteria group evaluation of variants under uncertainty.
• Verification looks for discrepancy between the expert knowledge and knowledge base.
• Experts and automatic verification system use the half-marks in inference rules.
• Example: assessing the effectiveness of a security screening system at an airport.

Knowledge engineering often involves using the opinions of experts, and very frequently of a group of experts. Experts often cooperate in creating a knowledge base that uses fuzzy inference rules. On the one hand, this may lead to generating a higher quality knowledge base. But on the other hand, it may result in irregularities, for example, if one of the experts dominates the others. This paper addresses a research problem related to creating a method for automatic verification of inference rules. It would allow one to detect inconsistencies between the rules that have been developed and the actual knowledge of the group of experts. A method of multi-criteria group evaluation of variants under uncertainty was used for this purpose. This method utilises experts’ opinions on the importance of the premises of inference rules. They are expressed in terms of multiple criteria in the form of both numerical and linguistic assessments. Experts define the conclusions of rules as so-called half-marks in order to increase the method’s flexibility. Automatic rules are generated in a similar way. Such an approach makes it possible to automatically determine the final conclusions of inference rules. They can be regarded as consistent both with the opinions of a group of experts and with automatically generated rules. This paper presents the use of the method for verifying the rules of an expert system that is aimed to evaluate the effectiveness of a passenger and baggage screening system at an airport. This method allows one to detect simple logical errors that are made when experts are establishing rules as well as inconsistencies between the rules that have been developed and the experts’ actual knowledge.