Automatic ink mismatch detection for forensic document analysis

•Ink mismatch detection identifies if part of a note was written with a different ink.•Apparently similar inks are automatically distinguished using spectral information.•A novel joint sparse band selection method is proposed for ink mismatch detection.•A new database is collected using our camera based document imaging system.•Extensive experiments are carried out to prove our claims.

A key aspect of handwritten document examination is to investigate whether some portion of the text was modified, altered or forged with a different pen. This paper demonstrates the use of hyperspectral imaging for ink mismatch detection in a handwritten note. We propose a novel joint sparse band selection technique that selects informative bands from hyperspectral images for accurate ink mismatch detection. We have developed an end-to-end camera-based hyperspectral document imaging system and collected a database of handwritten notes which has been made publicly available. Algorithmic solutions are presented to handle specific challenges in camera-based hyperspectral document imaging. Extensive experiments show that the proposed band selection method selects the most informative bands and improves average accuracy up to 15%, compared to using all bands.