Visual Analytics for model-based medical image segmentation: Opportunities and challenges

•We address model-based medical image segmentation from the Visual Analytics perspective.•Expert segmentations are costly and time consuming, current automatic segmentations are not precise enough. Better segmentation algorithms are needed; they can be created with help of Visual Analytics tools.•We identify four stages of the modeling process and present Visual Analytics methods for them.•They improve the final result by better setting of the algorithm parameters, analysis of the input data set, and allowing visual assessment of output quality on local level.•Based on these first results, we discussed what we see as most relevant challenges for the community.

Segmentation of medical images is a prerequisite in clinical practice. Many segmentation algorithms use statistical shape models. Due to the lack of tools providing prior information on the data, standard models are frequently used. However, they do not necessarily describe the data in an optimal way. Model-based segmentation can be supported by Visual Analytics tools, which give the user a deeper insight into the correspondence between data and model result. Combining both approaches, better models for segmentation of organs in medical images are created.In this work, we identify the main tasks and problems in model-based image segmentation. As a proof of concept, we show that already small visual-interactive extensions can be very beneficial. Based on these results, we present research challenges for Visual Analytics in this area.