A methodology for evaluating the performance of model-based traffic prediction systems

Model-based traffic prediction systems (mbTPS) are a central component of the decision support and ICM (integrated corridor management) systems currently used in several large urban traffic management centers. These models are intended to generate real-time predictions of the system's response to candidate operational interventions. They must therefore be kept calibrated and trustworthy. The methodologies currently available for tracking the validity of a mbTPS have been adapted from approaches originally designed for off-line operational planning models. These approaches are insensitive to the complexity of the network and to the amount and quality of the data available. They also require significant human intervention and are therefore not suitable for real-time monitoring. This paper outlines a set of criteria for designing tests that are appropriate for the mbTPS task. It also proposes a test that meets the criteria. The test compares the predictions of the mbTPS in question to those of a model-less alternative. A t-test is used to determine whether the predictions of the mbTPS are superior to those of the model-less predictor. The approach is applied to two different systems using data from the I-210 freeway in Southern California.