Unfolding the phenomenon of interrater agreement: a multicomponent approach for in-depth examination was proposed

Supported by a conceptual analysis, we propose an approach of in-depth examination of agreement variation, as a strategy for increasing the level of interrater agreement. By identifying and limiting the most important sources of disagreement, instrument reliability can be improved ultimately.