Dual-loop iterative optimal control for the finite horizon LQR problem with unknown dynamics

Achieving optimal performance over a finite-time horizon has gained a lot of attention in many engineering applications. Among them, the Finite Horizon Linear Quadratic Regulator (FHLQR) formulation for continuous-time linear time-varying systems has been well studied, with an optimal solution characterised by the Differential Riccati Equation (DRE). The solution of the DRE requires that the exact system dynamics are known. However, this assumption may not always hold, as the plant model might not be completely known or may change over time due to wear and tear. This paper proposes a dual-loop iterative algorithm to find the optimal solutions of the FHLQR formulation for continuous-time LTV systems. The inner loop utilises input trajectories based on an estimate of the optimal control gain with the addition of some excitation noise, and produces measured state trajectories. The outer loop improves the estimate of the optimal control gain utilising these measured state trajectories. It is shown in this work that with appropriate selection of the discretisation parameter T and the set of excitation signals, the proposed dual-loop iterative algorithm can converge to an arbitrarily small neighbourhood of the optimal solution. A simulation example demonstrates the effectiveness of the proposed method.