Information retrieval performance measures for a current awareness report composition aid

This papers studies a special “small” information retrieval problem where user satisfaction only depends on the ordering of documents. We look for a retrieval performance measure applicable for this setting. We define some requirements for such a measure. We develop a theoretical ordering of all outcomes. We look at some standard and purpose-build measures and assess them against the requirements. We conclude that a linear combination of two such measures is adequate.