Using chi-square statistics to measure similarities for text categorization

In this paper, we propose using chi-square statistics to measure similarities and chi-square tests to determine the homogeneity of two random samples of term vectors for text categorization. The properties of chi-square tests for text categorization are studied first. One of the advantages of chi-square test is that its significance level is similar to the miss rate that provides a foundation for theoretical performance (i.e. miss rate) guarantee. Generally a classifier using cosine similarities with TF ∗ IDF performs reasonably well in text categorization. However, its performance may fluctuate even near the optimal threshold value. To improve the limitation, we propose the combined usage of chi-square statistics and cosine similarities. Extensive experiment results verify properties of chi-square tests and performance of the combined usage.

Research highlights
► For a text categorization task, chi-square statistics can be used to measure dissimilarities and chi-square tests can be used as classifiers.
► The significance level of a chi-square test and the miss rate for the corresponding text categorization task are completely positive correlated.
► A chi-square test can determine the homogeneity of two random samples of original TF vectors without difficulty.
► A classifier using both cosine similarities with TF ∗ IDF and chi-square statistics as its similarity measures performs in par or better than one using only cosine similarity with TF ∗ IDF in F1.