Face recognition by fusing thermal infrared and visible imagery

Thermal infrared (IR) imagery offers a promising alternative to visible imagery for face recognition due to its relative insensitive to variations in face appearance caused by illumination changes. Despite its advantages, however, thermal IR has several limitations including that it is opaque to glass. The focus of this study is on the sensitivity of thermal IR imagery to facial occlusions caused by eyeglasses. Specifically, our experimental results illustrate that recognition performance in the IR spectrum degrades seriously when eyeglasses are present in the probe image but not in the gallery image and vice versa. To address this serious limitation of IR, we propose fusing IR with visible imagery. Since IR and visible imagery capture intrinsically different characteristics of the observed faces, intuitively, a better face description could be found by utilizing the complimentary information present in the two spectra. Two different fusion schemes have been investigated in this study. The first one is pixel-based and operates in the wavelet domain, while the second one is feature-based and operates in the eigenspace domain. In both cases, we employ a simple and general framework based on Genetic Algorithms (GAs) to find an optimum fusion strategy. We have evaluated our approaches through extensive experiments using the Equinox face database and the eigenface recognition methodology. Our results illustrate significant performance improvements in recognition, suggesting that IR and visible fusion is a viable approach that deserves further consideration.