An interactive and user-centered computer system to predict physician’s disease judgments in discharge summaries

Purpose: This article describes a formative natural language processing (NLP) system that is grounded in user-centered design, simplification, and transparency of function. Methods: The NLP system was tasked to classify diseases within patient discharge summaries and is evaluated against clinician judgment during the 2008 i2b2 Shared Task competition. Text classification is performed by interactive, fully supervised learning using rule-based processes and support vector machines (SVMs). Results: The macro-averaged F-score for textual (t) and intuitive (i) classification were 0.614(t) and 0.629(i), while micro-averaged F-scores were recorded at 0.966(t) and 0.954(i) for the competition. These results were comparable to the top 10 performing systems. Discussion: The results of this study indicate that an interactive training method, de novo knowledge base with no external data sources, and simplified text mining processes can achieve a comparably high performance in classifying health-related texts. Further research is needed to determine if the user-centered advantages of a NLP system translate into real world benefits.