Showing 1–4 of 4 results for author: Hendrickson, K

Distributed Nonconvex Optimization with Exponential Convergence Rate via Hybrid Systems Methods

Authors: Katherine R. Hendrickson, Dawn M. Hustig-Schultz, Matthew T. Hale, Ricardo G. Sanfelice

Abstract: We present a hybrid systems framework for distributed multi-agent optimization in which agents execute computations in continuous time and communicate in discrete time. The optimization algorithm is analogous to a continuous-time form of parallelized coordinate descent. Agents implement an update-and-hold strategy in which gradients are computed at communication times and held constant during flow… ▽ More We present a hybrid systems framework for distributed multi-agent optimization in which agents execute computations in continuous time and communicate in discrete time. The optimization algorithm is analogous to a continuous-time form of parallelized coordinate descent. Agents implement an update-and-hold strategy in which gradients are computed at communication times and held constant during flows between communications. The completeness of solutions under these hybrid dynamics is established. Then, we prove that this system is globally exponentially stable to a minimizer of a possibly nonconvex, smooth objective function that satisfies the Polyak-Lojasiewicz (PL) condition. Simulation results are presented for four different applications and illustrate the convergence rates and the impact of initial conditions upon convergence. △ Less

Submitted 20 January, 2025; originally announced February 2025.

Comments: 26 pages, 4 figures, currently under review

Totally Asynchronous Primal-Dual Convex Optimization in Blocks

Authors: Katherine Hendrickson, Matthew Hale

Abstract: We present a parallelized primal-dual algorithm for solving constrained convex optimization problems. The algorithm is "block-based," in that vectors of primal and dual variables are partitioned into blocks, each of which is updated only by a single processor. We consider four possible forms of asynchrony: in updates to primal variables, updates to dual variables, communications of primal variable… ▽ More We present a parallelized primal-dual algorithm for solving constrained convex optimization problems. The algorithm is "block-based," in that vectors of primal and dual variables are partitioned into blocks, each of which is updated only by a single processor. We consider four possible forms of asynchrony: in updates to primal variables, updates to dual variables, communications of primal variables, and communications of dual variables. We construct a family of explicit counterexamples to show the need to eliminate asynchronous communication of dual variables, though the other forms of asynchrony are permitted, all without requiring bounds on delays. A first-order primal-dual update law is developed and shown to be robust to asynchrony. We then derive convergence rates to a Lagrangian saddle point in terms of the operations agents execute, without specifying any timing or pattern with which they must be executed. These convergence rates include an "asynchrony penalty" that we quantify and present ways to mitigate. Numerical results illustrate these developments. △ Less

Submitted 3 May, 2022; v1 submitted 21 July, 2021; originally announced July 2021.

Comments: arXiv admin note: text overlap with arXiv:2004.05142

Exponentially Converging Distributed Gradient Descent with Intermittent Communication via Hybrid Methods

Authors: Katherine Hendrickson, Dawn Hustig-Schultz, Matthew Hale, Ricardo G. Sanfelice

Abstract: We present a hybrid systems framework for multi-agent optimization in which agents execute computations in continuous time and communicate in discrete time. The optimization algorithm is a hybrid version of parallelized coordinate descent. Agents implement a sample-and-hold strategy in which gradients are computed at communication times and held constant during flows between communications. Comple… ▽ More We present a hybrid systems framework for multi-agent optimization in which agents execute computations in continuous time and communicate in discrete time. The optimization algorithm is a hybrid version of parallelized coordinate descent. Agents implement a sample-and-hold strategy in which gradients are computed at communication times and held constant during flows between communications. Completeness of maximal solutions under these hybrid dynamics is established. Under assumptions of smoothness and strong convexity, we show that this system exponentially converges to the minimizer of an objective function. Simulation results illustrate this convergence rate. △ Less

Submitted 30 September, 2021; v1 submitted 20 April, 2021; originally announced April 2021.

Towards Totally Asynchronous Primal-Dual Convex Optimization in Blocks

Authors: Katherine Hendrickson, Matthew Hale

Abstract: We present a parallelized primal-dual algorithm for solving constrained convex optimization problems. The algorithm is "block-based," in that vectors of primal and dual variables are partitioned into blocks, each of which is updated only by a single processor. We consider four possible forms of asynchrony: in updates to primal variables, updates to dual variables, communications of primal variable… ▽ More We present a parallelized primal-dual algorithm for solving constrained convex optimization problems. The algorithm is "block-based," in that vectors of primal and dual variables are partitioned into blocks, each of which is updated only by a single processor. We consider four possible forms of asynchrony: in updates to primal variables, updates to dual variables, communications of primal variables, and communications of dual variables. We explicitly construct a family of counterexamples to rule out permitting asynchronous communication of dual variables, though the other forms of asynchrony are permitted, all without requiring bounds on delays. A first-order update law is developed and shown to be robust to asynchrony. We then derive convergence rates to a Lagrangian saddle point in terms of the operations agents execute, without specifying any timing or pattern with which they must be executed. These convergence rates contain a synchronous algorithm as a special case and are used to quantify an "asynchrony penalty." Numerical results illustrate these developments. △ Less

Submitted 28 August, 2020; v1 submitted 10 April, 2020; originally announced April 2020.