On signal representations within the Bayes decision framework

This work presents new results in the context of minimum probability of error signal representation (MPE-SR) within the Bayes decision framework. These results justify addressing the MPE-SR criterion as a complexity-regularized optimization problem, demonstrating the empirically well understood trade-off between signal representation quality and learning complexity. Contributions are presented in three folds. First, the stipulation of conditions that guarantee a formal tradeoff between approximation and estimation errors under sequence of embedded transformations are provided. Second, the use of this tradeoff to formulate the MPE-SR as a complexity regularized optimization problem, and an approach to address this oracle criterion in practice is given. Finally, formal connections are provided between the MPE-SR criterion and two emblematic feature transformation techniques used in pattern recognition: the optimal quantization problem of classification trees (CART tree pruning algorithms), and some versions of Fisher linear discriminant analysis (LDA).

► A formal tradeoff between Bayes error and estimation is presented. ► The tradeoff adopted to formulate the minimum probability error signal representation (MPE-SR). ► A practical complexity-regularized problem is proposed for addressing the MPE-SR. ► The CART pruning algorithm and Fisher linear discriminant are shown to be instances of the MPE-SR.