Models developed by three techniques did not achieve acceptable prediction of binary trauma outcomes

Background and ObjectivesTo develop prediction models for outcomes following trauma that met prespecified performance criteria. To compare three methods of developing prediction models: logistic regression, classification trees, and artificial neural networks.MethodsModels were developed using a 1996–2001 dataset from a major trauma center in Victoria, Australia. Developed models were subjected to external validation using the first year of data collection, 2001–2002, from a state-wide trauma registry for Victoria. Different authors developed models for each method. All authors were blinded to the validation dataset when developing models.ResultsPrediction models were developed for an intensive care unit stay following trauma (prevalence 23%) using information collected at the scene of the injury. None of the three methods gave a model that satisfied the performance criteria of sensitivity >80%, positive predictive value >50% in the validation dataset. Prediction models were also developed for death (prevalence 2.9%) using hospital-collected information. The performance criteria of sensitivity >95%, specificity >20% in the validation dataset were not satisfied by any model.ConclusionNo statistical method of model development was optimal. Prespecified performance criteria provide useful guides to interpreting the performance of developed models.