Criterion of calibration for transductive confidence machine with limited feedback

This paper is concerned with the problem of on-line prediction in the situation where some data are unlabelled and can never be used for prediction, and even when the data are labelled, the labels may arrive with a delay. We construct a modification of randomised transductive confidence machine for this case and prove a necessary and sufficient condition for its predictions being calibrated, in the sense that in the long run they are wrong with a prespecified probability under the assumption that the data are generated independently by the same distribution. The condition for calibration turns out to be very weak: feedback should be given on more than a logarithmic fraction of steps.