Implicit surrogate models for trust region based methods

In this paper, theoretical and practical aspects of Taylor based surrogate models for trust region (TR) optimization are discussed. We propose two surrogate models: one of them based on convex combinations of first order Taylor polynomials while the other one is based on that of quadratic Taylor expansions. During optimization steps of TR methods, the proposed models are built as follows: samples from the cost function are taken about the current solution and then, a linear or a quadratic Taylor polynomial is built about each sampled point. All Taylor expansions are convexly combined by using radial basis functions in order to propose a surrogate model for the optimization step. Theoretically, we prove that, the proposed models are fully linear or fully quadratic depending on whether they are built based on convex combinations of linear or quadratic Taylor expansions, respectively. This guaranties that, regardless the initial guess of the iterative process, the optimization method will converge to local solutions where at least first order optimality conditions are satisfied. Discussions on how to solve the surrogate optimization problems among iterations are presented as well. Experimental results are carried out making use of a sequential data assimilation problem. The results reveal that, the use of our proposed models in such context can improve the quality of analysis states in terms of root-mean-square-error values while uncertainties can be decreased across different assimilation cycles.