Automatic ink mismatch detection for forensic document analysis

• Ink mismatch detection identifies if part of a note was written with a different ink.
• Apparently similar inks are automatically distinguished using spectral information.
• A novel joint sparse band selection method is proposed for ink mismatch detection.
• A new database is collected using our camera based document imaging system.
• Extensive experiments are carried out to prove our claims.

A key aspect of handwritten document examination is to investigate whether some portion of the text was modified, altered or forged with a different pen. This paper demonstrates the use of hyperspectral imaging for ink mismatch detection in a handwritten note. We propose a novel joint sparse band selection technique that selects informative bands from hyperspectral images for accurate ink mismatch detection. We have developed an end-to-end camera-based hyperspectral document imaging system and collected a database of handwritten notes which has been made publicly available. Algorithmic solutions are presented to handle specific challenges in camera-based hyperspectral document imaging. Extensive experiments show that the proposed band selection method selects the most informative bands and improves average accuracy up to 15%, compared to using all bands.