A method for determining the number of documents needed for a gold standard corpus

► Annotated documents are necessary for NLP machine learning, modeling and testing. ► We create a method to determine a required sample size for the annotation set. ► The probability of word capture from a corpus provides the basis for the method. ► Dictation letters from a pain management medical practice are used as an example. ► We also demonstrate steps for creating a representative sample of dictations.