Concept drift mining of portfolio selection factors in stock market

•Provide a model based on causal discovery technique (ANMCPT) for concept drift mining in cross-sectional analysis.•AVMCPT can discover causal in high-dimensional and dynamic environment.•ANMCPT outperform the classical Fama–French framework.•Concept drift phenomenon in China stock market is observed and exhibited clearly.

Concept drift is a common phenomenon in stock market that can cause the devaluation of the knowledge learned from cross-sectional analysis as the market changes over time in unforeseen ways. The widely used cross-sectional regression analysis based on expert knowledge has obvious limitations in handling problems that involve concept drift and high-dimensional data. To discover causal relations between portfolio selection factors and stock returns, and identify concept drifts of these relations, we apply a novel causal discovery technique called modified Additive Noise Model with Conditional Probability Table (ANMCPT). In evaluation experiments, we compares ANMCPT to the conventional cross-sectional analysis approach (i.e., Fama–French framework) in mining relationships between portfolio selection factors and stock returns. Results indicate that the factors selected by ANMCPT outperform the factors adopted in most previous cross-sectional researches that followed the Fama–French framework. To the best of our knowledge, this paper is the first to compare causal inference technique with Fama–French framework in concept drift mining of stock portfolio selection factors. Our causal inference-based concept drift mining method provides a new approach to accurate knowledge discovery in stock market.