-
Exact Verification of First-Order Methods via Mixed-Integer Linear Programming
Authors:
Vinit Ranjan,
Jisun Park,
Stefano Gualandi,
Andrea Lodi,
Bartolomeo Stellato
Abstract:
We present exact mixed-integer linear programming formulations for verifying the performance of first-order methods for parametric quadratic optimization. We formulate the verification problem as a mixed-integer linear program where the objective is to maximize the infinity norm of the fixed-point residual after a given number of iterations. Our approach captures a wide range of gradient, projecti…
▽ More
We present exact mixed-integer linear programming formulations for verifying the performance of first-order methods for parametric quadratic optimization. We formulate the verification problem as a mixed-integer linear program where the objective is to maximize the infinity norm of the fixed-point residual after a given number of iterations. Our approach captures a wide range of gradient, projection, proximal iterations through affine or piecewise affine constraints. We derive tight polyhedral convex hull formulations of the constraints representing the algorithm iterations. To improve the scalability, we develop a custom bound tightening technique combining interval propagation, operator theory, and optimization-based bound tightening. Numerical examples, including linear and quadratic programs from network optimization, sparse coding using Lasso, and optimal control, show that our method provides several orders of magnitude reductions in the worst-case fixed-point residuals, closely matching the true worst-case performance.
△ Less
Submitted 23 May, 2025; v1 submitted 15 December, 2024;
originally announced December 2024.
-
Verification of First-Order Methods for Parametric Quadratic Optimization
Authors:
Vinit Ranjan,
Bartolomeo Stellato
Abstract:
We introduce a numerical framework to verify the finite step convergence of first-order methods for parametric convex quadratic optimization. We formulate the verification problem as a mathematical optimization problem where we maximize a performance metric (e.g., fixed-point residual at the last iteration) subject to constraints representing proximal algorithm steps (e.g., linear system solutions…
▽ More
We introduce a numerical framework to verify the finite step convergence of first-order methods for parametric convex quadratic optimization. We formulate the verification problem as a mathematical optimization problem where we maximize a performance metric (e.g., fixed-point residual at the last iteration) subject to constraints representing proximal algorithm steps (e.g., linear system solutions, projections, or gradient steps). Our framework is highly modular because we encode a wide range of proximal algorithms as variations of two primitive steps: affine steps and element-wise maximum steps. Compared to standard convergence analysis and performance estimation techniques, we can explicitly quantify the effects of warm-starting by directly representing the sets where the initial iterates and parameters live. We show that the verification problem is NP-hard, and we construct strong semidefinite programming relaxations using various constraint tightening techniques. Numerical examples in nonnegative least squares, network utility maximization, Lasso, and optimal control show a significant reduction in pessimism of our framework compared to standard worst-case convergence analysis techniques.
△ Less
Submitted 16 April, 2025; v1 submitted 5 March, 2024;
originally announced March 2024.
-
Block size in Geometric(p)-biased permutations
Authors:
Irina Cristali,
Vinit Ranjan,
Jake Steinberg,
Erin Beckman,
Rick Durrett,
Matthew Junge,
James Nolen
Abstract:
Fix a probability distribution $\mathbf p = (p_1, p_2, \cdots)$ on the positive integers. The first block in a $\mathbf p$-biased permutation can be visualized in terms of raindrops that land at each positive integer $j$ with probability $p_j$. It is the first point $K$ so that all sites in $[1,K]$ are wet and all sites in $(K,\infty)$ are dry. For the geometric distribution $p_j= p(1-p)^{j-1}$ we…
▽ More
Fix a probability distribution $\mathbf p = (p_1, p_2, \cdots)$ on the positive integers. The first block in a $\mathbf p$-biased permutation can be visualized in terms of raindrops that land at each positive integer $j$ with probability $p_j$. It is the first point $K$ so that all sites in $[1,K]$ are wet and all sites in $(K,\infty)$ are dry. For the geometric distribution $p_j= p(1-p)^{j-1}$ we show that $p \log K$ converges in probability to an explicit constant as $p$ tends to 0. Additionally, we prove that if $\mathbf p$ has a stretch exponential distribution, then $K$ is infinite with positive probability.
△ Less
Submitted 13 August, 2018; v1 submitted 18 August, 2017;
originally announced August 2017.
