Multimodal biometric system built on the new entropy function for feature extraction and the Refined Scores as a classifier

•A new entropy that can modify the probabilities in the function is formulated.•Two information set based features: EGI and EEI are derived from the above function.•A unique multimodal biometric system comprising IR face, iris and ear is developed.•A classifier based on Refined Scores that use cohort scores in refining is devised.•The combined scores from individual modalities are fused at the score level fusion and then improved by RS method.

This paper presents a unique face based multimodal biometric system comprising IR face, ear and iris to cater to the surveillance application by proposing new entropy function. Two new features based on this entropy are devised to cater the highly uncertain database found at the surveillance site. To handle the erroneous scores we have proposed Refined Score (RS) method and applied it on individual IR face, ear and iris modalities under both constrained and the unconstrained conditions for the authentication of users and also used for the score level fusion of these modalities using the proposed entropy based features. The entropy features show good performance under the constrained and unconstrained databases whereas the conventional entropies do not fare well on the unconstrained databases. RS based classifier always outperforms the EC (Euclidean classifier) and RS based score level fusion has an edge over the conventional score level fusion.