File Formats - Characterization and Validation

Nowadays, most of the information is stored digitally. Digital information is from a high level of view it is just an array of bits. In order to figure out its real meaning special software which interprets it is required. Therefore, if by evolution of technology this software cannot be executed anymore there is potential risk that also the data interpreted by it becomes not useful. The goal of Digital Preservation is to stop occurrences of such phenomenon. Data is commonly stored in files each file has a specific format or structure, by knowing it user can figure out the real meaning of raw data stored in the file as an array of bits. Digital Preservation considers valid file format as a perquisite for file to be in usable form, with valid is meant that a specific file is structured conform its declared file format. In this paper we throw a spotlight on the accuracy and capability of these file validation tests. Therefore, we present some open source software which are able to automatically identify and verify the file format. We focus more on file types they can identify, and how they work in large scale data sets.