Semi-hidden target recognition in gated viewer images fused with thermal IR images

•Fusion of GV with IR imagery improves human recognition of semi-hidden targets.•A simple pixel-based PCA-based weighted fusion scheme appears the optimal fusion method.•Contextual dimming improves target recognition in complex backgrounds.•Moving targets require more dimming than static targets to obtain a given recognition level.

Defense and security surveillance scenarios typically involve the detection and classification of targets in complex and dynamic backgrounds. Imaging systems deployed for this purpose should therefore provide imagery that enables optimal simultaneous recognition of both targets and their context. Here we investigate the recognition of semi-hidden targets, which are targets that are embedded in complex scenes, and which may either be occluded by or merged with other details in the scene. Imagery of semi-hidden targets obtained with conventional visual (TV) and Infra-Red (IR) cameras is typically not optimal for recognition and classification purposes. Previous studies on image fusion did not consider semi-hidden targets. This study investigates the potential benefits of (1) adding a laser range gated viewer (GV) to an IR camera and of (2) fusing GV and IR imagery for the recognition of semi-hidden targets. A combination of an Image Quality Metric (IQM) and an accurate saliency metric is used to select a fusion method that is optimal for semi-hidden target recognition. The results of both metrics are validated through a human observer experiment. For application in very complex scenes (in which target recognition remains difficult after fusion) we designed a background dimming algorithm that either uniformly dims the entire background or applies less dimming in the local target background or in regions with important contextual information, without affecting the target representation itself. The optimal combination of fusion method and amount of dimming is determined through a second observer experiment. In a third observer experiment, we tested if target motion influences the preferred amount of dimming. We find that fusing GV with IR imagery improves human recognition of semi-hidden targets. A simple pixel-based approach with a PCA-based weighted fusion scheme appears to be the optimal fusion method. Contextual dimming improves target recognition in complex backgrounds. In addition, moving objects appear to affect observer’s dimming preference, but further research is needed to quantify this effect.