Interrater agreement in visual scoring of neonatal seizures based on majority voting on a web-based system: The Neoguard EEG database

•Majority voting in neonatal seizures can be used for clinical validation and to develop uniform definitions.•The duration of events and the composition of databases influence seizure recognition and agreement.•Well-described datasets help to improve EEG interpretation and serve as learning platform.

ObjectiveTo assess interrater agreement based on majority voting in visual scoring of neonatal seizures.MethodsAn online platform was designed based on a multicentre seizure EEG-database. Consensus decision based on 'majority voting' and interrater agreement was estimated using Fleiss' Kappa. The influences of different factors on agreement were determined.Results1919 Events extracted from 280 h EEG of 71 neonates were reviewed by 4 raters. Majority voting was applied to assign a seizure/non-seizure classification. 44% of events were classified with high, 36% with moderate, and 20% with poor agreement, resulting in a Kappa value of 0.39. 68% of events were labelled as seizures, and in 46%, all raters were convinced about electrographic seizures. The most common seizure duration was <30 s. Raters agreed best for seizures lasting 60-120 s. There was a significant difference in electrographic characteristics of seizures versus dubious events, with seizures having longer duration, higher power and amplitude.ConclusionsThere is a wide variability in identifying rhythmic ictal and non-ictal EEG events, and only the most robust ictal patterns are consistently agreed upon. Database composition and electrographic characteristics are important factors that influence interrater agreement.SignificanceThe use of well-described databases and input of different experts will improve neonatal EEG interpretation and help to develop uniform seizure definitions, useful for evidence-based studies of seizure recognition and management.