Design of second order neural networks as dynamical control systems that aim to minimize nonconvex scalar functions

This paper presents a unified way to design neural networks characterized as second order ordinary differential equations (ODEs) that aim to find the global minimum of nonconvex scalar functions. These neural networks, alternatively referred to as continuous time algorithms, are interpreted as dynamical closed loop control systems. The design is based on the control Liapunov function (CLF) method. For nonconvex scalar functions, the goal of these algorithms is to produce trajectories, starting from an arbitrarily chosen initial guess, that do not get stuck in local minima, thereby increasing the chances of converging to the global minimum.