Assessing semantic annotation activities with formal concept analysis

•Concept lattices are built from ontology-annotated resources by using FCA.•Domain experts use these lattices to assess annotation activities.•Experts can use lattices to help annotators make better use of ontologies.•Experts also can use lattices to refine their annotation ontologies.

This paper describes an approach to assessing semantic annotation activities based on formal concept analysis (FCA). In this approach, annotators use taxonomical ontologies created by domain experts to annotate digital resources. Then, using FCA, domain experts are provided with concept lattices that graphically display how their ontologies were used during the semantic annotation process. In consequence, they can advise annotators on how to better use the ontologies, as well as how to refine these ontologies to better suit the needs of the semantic annotators. To illustrate the approach, we describe its implementation in @note, a Rich Internet Application (RIA) for the collaborative annotation of digitized literary texts, we exemplify its use with a case study, and we provide some evaluation results using the method.