Bayesian model selection using automatic relevance determination for nonlinear dynamical systems

Bayesian model selection is augmented with automatic relevance determination (ARD) to perform model reduction of complex dynamical systems modelled by nonlinear, stochastic ordinary differential equations (ODE). Given noisy measurement data, a parametrically flexible model is envisioned to represent the dynamical system. A Bayesian model selection problem is posed to find the best model nested under the envisioned model. This model selection problem is transferred from the model space to hyper-parameter space by regularizing the parameter posterior space through a parametrized prior distribution called the ARD prior. The resulting joint prior pdf is the combination of parametrized ARD priors assigned to parameters whose relevance to the system dynamics is questionable and the known prior pdf for parameters whose relevance is known a priori. The hyper-parameter of each ARD prior explicitly represents the relevance of the corresponding model parameter. The hyper-parameters are estimated using the measurement data by performing evidence maximization or type-II maximum likelihood. Superfluous model parameters are switched off during evidence maximization by the corresponding ARD prior, forcing the model parameter to be irrelevant for prediction purposes. An efficient numerical implementation for evidence computation using Markov Chain Monte Carlo sampling of the parameter posterior distribution is presented for the case when the analytical evaluation of evidence is not possible. The ARD approach is validated with synthetic measurements generated from a nonlinear, unsteady aeroelastic oscillator consisting of a NACA0012 airfoil undergoing limit cycle oscillation. A set of intentionally flexible stochastic ODEs having different state-space formulation is proposed to model the synthetic data. ARD is used to obtain an optimal nested model corresponding to each proposed model. The optimal nested model with the maximum posterior model probability is chosen as the overall optimal model. ARD provides a flexible Bayesian platform to find the optimal nested model by eliminating the need to propose candidate nested models and its prior pdfs.