کد مقاله کد نشریه سال انتشار مقاله انگلیسی نسخه تمام متن
532308 869931 2013 11 صفحه PDF دانلود رایگان
عنوان انگلیسی مقاله ISI
The reliability of estimated confidence intervals for classification error rates when only a single sample is available
موضوعات مرتبط
مهندسی و علوم پایه مهندسی کامپیوتر چشم انداز کامپیوتر و تشخیص الگو
پیش نمایش صفحه اول مقاله
The reliability of estimated confidence intervals for classification error rates when only a single sample is available
چکیده انگلیسی

Error estimation accuracy is the salient issue regarding the validity of a classifier model. When samples are small, training-data-based error estimates tend to suffer from inaccuracy and quantification of error estimation accuracy is difficult. Numerous methods have been proposed for estimating confidence intervals for the true error based on the estimated error. This paper surveys proposed methods and quantifies their performance. Monte Carlo methods are used to obtain accurate estimates of the true confidence intervals and compare these to the intervals estimated from samples. We consider different error estimators and several proposed confidence-bound estimators. Both synthetic and real genomic data are employed. Our simulations show the majority of the confidence intervals methods have poor performance because of the difference of shape between true and estimated intervals. According to our results, the best estimation strategy is to use the 10-time 10-fold cross-validation with a confidence interval based on the standard deviation.


► We present different estimators of the confidence interval for the error rate estimation.
► Our experimentation shows which are the best confidence interval estimators.
► We show the lack of accuracy of most of the confidence interval estimators.

ناشر
Database: Elsevier - ScienceDirect (ساینس دایرکت)
Journal: Pattern Recognition - Volume 46, Issue 3, March 2013, Pages 1067–1077
نویسندگان
, ,