Improving the performance of question answering with semantically equivalent answer patterns

In this paper, we discuss a novel technique based on semantic constraints to improve the performance and portability of a reformulation-based question answering system. First, we present a method for acquiring semantic-based reformulations automatically. The goal is to generate patterns from sentences retrieved from the Web based on lexical, syntactic and semantic constraints. Once these constraints have been defined, we present a method to evaluate and re-rank candidate answers that satisfy these constraints using redundancy. The two approaches have been evaluated independently and in combination. The evaluation on 493 questions from TREC-11 shows that the automatically acquired semantic patterns increase the MRR by 26%, the re-ranking using semantic redundancy increases the MRR by 67%, and the two approaches combined increase the MRR by 73%. This new technique allows us to avoid the manual work of formulating semantically equivalent reformulations; while still increasing performance.