Evaluation of asymptotic approximations for a two-stage Bernoulli bandit

We consider a two-stage procedure for allocating two treatments to yield a total of N dichoto-mous responses, where one of the treatments has a known probability of success. In the first stage, observations may be made on either of the treatments and observed successes are discounted by a factor β. One of the treatments must be chosen for the second stage, where observed successes are no longer discounted. We adopt a Bayesian approach and develop a continuous time approximation for this problem that turns out to be identical to one developed in Petkau (J. Amer. Statist. Assoc. 73 (1978) 328). Examination of both stopping boundaries and Bayes risks demonstrates that suboptimal strategies provided by the solution of the continuous time problem are excellent approximations to the optimal strategies for the discrete time problem. A “continuity correction” developed by Cheroff and Petkau (Ann. Probab. 4 (1976) 875) plays an important role in enhancing the naive approximation provided by the solution of the continuous time problem.