Influence of pre-processing and distance on spectral classification: A simulation study

An investigation on the influence of pre-processing on the recognition of chemically similar areas in a spectral image, using simulated data. Fictitious spectra of mixtures of five components at varying concentrations were corrupted by different types of noise to mimic typical signals from Raman imaging. They were then processed by various combinations of pre-processing functions, including baseline correction, smoothing, normalization and Principal Components (PC) compression, and by two clustering algorithms (k-means and agglomerative hierarchical clustering) to recognize the original mixtures. The clusters obtained by the different pre-processing combinations and distance metrics were evaluated by statistical parameters (Rand index and silhouette coefficient) and visual inspection. Perhaps the best performing on the basis of all considered criteria is the combination using an adaptive polynomial detrending, a slight smoothing, normalization by the total signal intensity and compression by 4 PCs (spanning 80% of the total variance). More detailed analysis was also carried out on subsets of the whole data with a particular type of noise and on the influence of each single pre-processing/clustering variable.