Enhancement of digital reading performance by using a novel web-based collaborative reading annotation system with two quality annotation filtering mechanisms

• Proposing high-grade and master annotation filtering schemes for annotation systems.
• Reviewing annotation frequency is an effective means of identifying high-quality annotations.
• High-grade annotation filtering can improve reading comprehension.
• High-grade annotation filtering is recommended to identify quality annotations.
• Quality annotation filtering schemes do not significantly reduce cognitive load.

Collaboratively annotating digital texts allows learners to add valued information, share ideas, and create knowledge. However, excessive annotations and poor-quality annotations in a digital text may cause information overload and divert attention from the main content. The increased cognitive load ultimately reduces the effectiveness of collaborative annotations in promoting reading comprehension. Thus, this work develops a web-based collaborative reading annotation system (WCRAS-TQAFM) with two quality annotation filtering mechanisms—high-grade and master annotation filters—to promote the reading performance of learners. Ninety-seven students from three classes of a senior high school in Taiwan were invited to participate in an 80-min reading activity in which individual readers use WCRAS with or without annotation filters. Analytical results indicate that digital reading performance is significantly better in readers who use the high-grade annotation filter compared to those who read all annotations. Moreover, the high-grade annotation filter can enhance the reading comprehension of learners in all considered question types (i.e., recall, main idea, inference, and application). Also, the Cohen’s kappa statistics was used for assessing whether the annotation selected by the high-grade annotation filter is in agreement with the annotations selected by a domain expert. The statistic results indicate that the proposed high-grade annotation filter is valid to some degree. Finally, neither of the proposed quality annotation filtering approaches significantly reduces cognitive load.