Combining Multiple Measures into a Single Figure of Merit

Researchers often collect a number of dependent measures in a study, each of which may reflect some aspect of overall performance. In psychology, in particular response time and accuracy are frequently used measures. Trade-offs between speed and accuracy are often observed, and can occur between other measures. How should overall performance be characterized in the presence of trade-offs? In this paper, we consider how multiple measures can be combined into a single, overall measure of performance. We consider two different case studies, one involving speed and accuracy, and the other involving the relationship between sampling of visual information and resulting accuracy in a pedal-tracking task. We define a global measure of performance using summated z-scores. We discuss the behavior of this measure and contrast it with other approaches to creating linear combinations of variables.