Feature evaluation for web crawler detection with data mining techniques

► We examine the effect of applying 7 data mining classifiers on web server logs. ► We automate the detection of well-behaved and malicious crawlers. ► We automate the detection of human and unknown visitors. ► We introduce 2 novel features to facilitate classification of web site visitors. ► Results show improvement in recall and precision of detection.