Finite-sample density and its small sample asymptotic approximation

To derive the exact density of a statistic, which can be intractable, is sometimes a difficult problem. The exact densities of estimates of the shift or regression parameters can be derived with the aid of score functions. Moreover, extremely accurate approximations can be obtained by the small sample asymptotics, based on the saddlepoint method. It is of interest to compare these two approaches, at least for small samples. We numerically compare the exact densities of estimates of the shift parameter with their small sample approximations for various parent distributions of the data. For some distributions both methods are in surprising concordance even under very small samples.