Evaluation of brain atrophy estimation algorithms using simulated ground-truth data

A number of analysis tools have been developed for the estimation of brain atrophy using MRI. Since brain atrophy is being increasingly used as a marker of disease progression in many neuro-degenerative diseases such as Multiple Sclerosis and Alzheimer's disease, the validation of these tools is an important task. However, this is complex, in the real scenario, due to the absence of gold standards for comparison. In order to create gold standards, we first propose an approach for the realistic simulation of brain tissue loss that relies on the estimation of a topology preserving B-spline based deformation fields. Using these gold standards, an evaluation of the performance of three standard brain atrophy estimation methods (SIENA, SIENAX and BSI-UCD), on the basis of their robustness to various sources of error (bias-field inhomogeneity, noise, geometrical distortions, interpolation artefacts and presence of lesions), is presented. Our evaluation shows that, in general, bias-field inhomogeneity and noise lead to larger errors in the estimated atrophy than geometrical distortions and interpolation artefacts. Experiments on 18 different anatomical models of the brain after simulating whole brain atrophies in the range of 0.2-1.5% indicate that, in the presence of bias-field inhomogeneity and noise, a mean error of 0.64±0.53%,4.00±2.41% and 1.79±0.97% may be expected in the atrophy estimated by SIENA, SIENAX and BSI-UCD, respectively.