Evaluation of the performances of different P300 based brain–computer interfaces by means of the efficiency metric

The aim of this paper is to show how to use the Efficiency, a brain–computer interface (BCI) performance indicator, to evaluate the performances of a wide range of BCI systems. Unlike the most used metrics in the BCI research field, the Efficiency takes into account the penalties and the strategies to recover errors and this makes it a reliable instrument to describe the behavior of real BCIs. The Efficiency is compared with the accuracy and the information transfer rate, both in the Wolpaw and Nykopp definitions. The comparison covers four widely used classifiers and different stimulation sequences.Results show that the Efficiency is able to predict if the communication will not be possible, because the time spent to correct mistakes is longer than the time needed to generate a correct selection, and therefore it provides a much more realistic evaluation of a system. It can also be easily adapted to evaluate different applications, so it reveals a more general and versatile indicator for BCI systems.

► The efficiency metric has been used to evaluate different configurations of BCI systems.
► It was compared with accuracy and information transfer rate.
► Results show that efficiency is a better and more realistic metric because it can predict when the communication is not possible.
► The efficiency should be used for the optimization of BCI systems.