Free-form image registration regularized by a statistical shape model: application to organ segmentation in cervical MR

•We propose a registration scheme that is enhanced by a statistical model.•Registration is performed on MR images of 17 patients with cervical cancer.•Compared to conventional registration segmentation of the evaluated structure is improved.

Deformable registration is prone to errors when it involves large and complex deformations, since the procedure can easily end up in a local minimum. To reduce the number of local minima, and thus the risk of misalignment, regularization terms based on prior knowledge can be incorporated in registration. We propose a regularization term that is based on statistical knowledge of the deformations that are to be expected. A statistical model, trained on the shapes of a set of segmentations, is integrated as a penalty term in a free-form registration framework. For the evaluation of our approach, we perform inter-patient registration of MR images, which were acquired for planning of radiation therapy of cervical cancer. The manual delineations of structures such as the bladder and the clinical target volume are available. For both structures, leave-one-patient-out registration experiments were performed. The propagated atlas segmentations were compared to the manual target segmentations by Dice similarity and Hausdorff distance. Compared with registration without the use of statistical knowledge, the segmentations were significantly improved, by 0.1 in Dice similarity and by 8 mm Hausdorff distance on average for both structures.