The Controlled Conjugate Gradient Trajectory-Following Algorithm for Minimization of Non-convex Functions

Abstract: The paper presents a unified way to design second-order ordinary differential equations (ODEs) in such a way that their trajectories can converge to the global minimum of non-convex scalar functions. These ODEs, sometimes called continuous-time algorithms, are interpreted as closed-loop control systems and the design is based on a tool known as a control Liapunov function. For non-convex functions, the goal of these algorithms is to produce continuous-time trajectories, that start from an arbitrary initial point and can escape from local minima, thereby increasing the chances of converging to the global minimum. The focus will be on a new controlled conjugate gradient family of algorithms, derived from the well known discrete conjugate gradient algorithm and a generalization of the so called Heavy Ball with Friction family of algorithms.