-
An Interior-Point Algorithm for Continuous Nonlinearly Constrained Optimization with Noisy Function and Derivative Evaluations
Authors:
Frank E. Curtis,
Shima Dezfulian,
Andreas Waechter
Abstract:
An algorithm based on the interior-point methodology for solving continuous nonlinearly constrained optimization problems is proposed, analyzed, and tested. The distinguishing feature of the algorithm is that it presumes that only noisy values of the objective and constraint functions and their first-order derivatives are available. The algorithm is based on a combination of a previously proposed…
▽ More
An algorithm based on the interior-point methodology for solving continuous nonlinearly constrained optimization problems is proposed, analyzed, and tested. The distinguishing feature of the algorithm is that it presumes that only noisy values of the objective and constraint functions and their first-order derivatives are available. The algorithm is based on a combination of a previously proposed interior-point algorithm that allows inexact subproblem solutions and recently proposed algorithms for solving bound- and equality-constrained optimization problems with only noisy function and derivative values. It is shown that the new interior-point algorithm drives a stationarity measure below a threshold that depends on bounds on the noise in the function and derivative values. The results of numerical experiments show that the algorithm is effective across a wide range of problems.
△ Less
Submitted 16 February, 2025;
originally announced February 2025.
-
A Decomposition Framework for Nonlinear Nonconvex Two-Stage Optimization
Authors:
Yuchen Lou,
Xinyi Luo,
Andreas Wächter,
Ermin Wei
Abstract:
We propose a new decomposition framework for continuous nonlinear constrained two-stage optimization, where both first- and second-stage problems can be nonconvex. A smoothing technique based on an interior-point formulation renders the optimal solution of the second-stage problem differentiable with respect to the first-stage parameters. As a consequence, efficient off-the-shelf optimization pack…
▽ More
We propose a new decomposition framework for continuous nonlinear constrained two-stage optimization, where both first- and second-stage problems can be nonconvex. A smoothing technique based on an interior-point formulation renders the optimal solution of the second-stage problem differentiable with respect to the first-stage parameters. As a consequence, efficient off-the-shelf optimization packages can be utilized. We show that the solution of the nonconvex second-stage problem behaves locally like a differentiable function so that existing proofs can be applied for the global convergence of the first-stage. We also prove fast local convergence of the algorithm as the barrier parameter is driven to zero. Numerical experiments for large-scale instances demonstrate the computational advantages of the decomposition framework.
△ Less
Submitted 20 January, 2025;
originally announced January 2025.
-
On the Convergence of Interior-Point Methods for Bound-Constrained Nonlinear Optimization Problems with Noise
Authors:
Shima Dezfulian,
Andreas Wächter
Abstract:
We analyze the convergence properties of a modified barrier method for solving bound-constrained optimization problems where evaluations of the objective function and its derivatives are affected by bounded and non-diminishing noise. The only modification compared to a standard barrier method is a relaxation of the Armijo line-search condition. We prove that the algorithm generates iterates at whi…
▽ More
We analyze the convergence properties of a modified barrier method for solving bound-constrained optimization problems where evaluations of the objective function and its derivatives are affected by bounded and non-diminishing noise. The only modification compared to a standard barrier method is a relaxation of the Armijo line-search condition. We prove that the algorithm generates iterates at which the size of the barrier function gradient eventually falls below a threshold that converges to zero if the noise level converges to zero. Based on this result, we propose a practical stopping test that does not require estimates of unknown problem parameters and identifies iterations in which the theoretical threshold is reached. We also analyze the local convergence properties of the method when noisy second derivatives are used. Under a strict-complementarity assumption, we show that iterates stay in a neighborhood around the optimal solution once it is entered. The neighborhood is defined in a scaled norm that becomes narrower for variables with active bound constraints as the barrier parameter is decreased. As a consequence, we show that active bound constraints can be identified despite noise. Numerical results demonstrate the effectiveness of the stopping test and illustrate the active-set identification properties of the method.
△ Less
Submitted 18 May, 2024;
originally announced May 2024.
-
Getting to "rate-optimal'' in ranking & selection
Authors:
Harun Avci,
Barry L. Nelson,
Andreas Wächter
Abstract:
In their 2004 seminal paper, Glynn and Juneja formally and precisely established the rate-optimal, probability-of-incorrect-selection, replication allocation scheme for selecting the best of k simulated systems. In the case of independent, normally distributed outputs this allocation has a simple form that depends in an intuitively appealing way on the true means and variances. Of course the means…
▽ More
In their 2004 seminal paper, Glynn and Juneja formally and precisely established the rate-optimal, probability-of-incorrect-selection, replication allocation scheme for selecting the best of k simulated systems. In the case of independent, normally distributed outputs this allocation has a simple form that depends in an intuitively appealing way on the true means and variances. Of course the means and (typically) variances are unknown, but the rate-optimal allocation provides a target for implementable, dynamic, data-driven policies to achieve. In this paper we compare the empirical behavior of four related replication-allocation policies: mCEI from Chen and Rzyhov and our new gCEI policy that both converge to the Glynn and Juneja allocation; AOMAP from Peng and Fu that converges to the OCBA optimal allocation; and TTTS from Russo that targets the rate of convergence of the posterior probability of incorrect selection. We find that these policies have distinctly different behavior in some settings.
△ Less
Submitted 4 February, 2023;
originally announced February 2023.
-
A Quadratically Convergent Sequential Programming Method for Second-Order Cone Programs Capable of Warm Starts
Authors:
Xinyi Luo,
Andreas Waechter
Abstract:
We propose a new method for linear second-order cone programs. It is based on the sequential quadratic programming framework for nonlinear programming. In contrast to interior point methods, it can capitalize on the warm-start capabilities of active-set quadratic programming subproblem solvers and achieve a local quadratic rate of convergence.
In order to overcome the non-differentiability or si…
▽ More
We propose a new method for linear second-order cone programs. It is based on the sequential quadratic programming framework for nonlinear programming. In contrast to interior point methods, it can capitalize on the warm-start capabilities of active-set quadratic programming subproblem solvers and achieve a local quadratic rate of convergence.
In order to overcome the non-differentiability or singularity observed in nonlinear formulations of the conic constraints, the subproblems approximate the cones with polyhedral outer approximations that are refined throughout the iterations. For nondegenerate instances, the algorithm implicitly identifies the set of cones for which the optimal solution lies at the extreme points. As a consequence, the final steps are identical to regular sequential quadratic programming steps for a differentiable nonlinear optimization problem, yielding local quadratic convergence.
We prove the global and local convergence guarantees of the method and present numerical experiments that confirm that the method can take advantage of good starting points and can achieve higher accuracy compared to a state-of-the-art interior point solver.
△ Less
Submitted 28 July, 2023; v1 submitted 7 July, 2022;
originally announced July 2022.
-
Recent Developments in Security-Constrained AC Optimal Power Flow: Overview of Challenge 1 in the ARPA-E Grid Optimization Competition
Authors:
Ignacio Aravena,
Daniel K. Molzahn,
Shixuan Zhang,
Cosmin G. Petra,
Frank E. Curtis,
Shenyinying Tu,
Andreas Wächter,
Ermin Wei,
Elizabeth Wong,
Amin Gholami,
Kaizhao Sun,
Xu Andy Sun,
Stephen T. Elbert,
Jesse T. Holzer,
Arun Veeramany
Abstract:
The optimal power flow problem is central to many tasks in the design and operation of electric power grids. This problem seeks the minimum cost operating point for an electric power grid while satisfying both engineering requirements and physical laws describing how power flows through the electric network. By additionally considering the possibility of component failures and using an accurate AC…
▽ More
The optimal power flow problem is central to many tasks in the design and operation of electric power grids. This problem seeks the minimum cost operating point for an electric power grid while satisfying both engineering requirements and physical laws describing how power flows through the electric network. By additionally considering the possibility of component failures and using an accurate AC power flow model of the electric network, the security-constrained AC optimal power flow (SC-AC-OPF) problem is of paramount practical relevance. To assess recent progress in solution algorithms for SC-AC-OPF problems and spur new innovations, the U.S. Department of Energy's Advanced Research Projects Agency--Energy (ARPA-E) organized Challenge 1 of the Grid Optimization (GO) competition. This paper describes the SC-AC-OPF problem formulation used in the competition, overviews historical developments and the state of the art in SC-AC-OPF algorithms, discusses the competition, and summarizes the algorithms used by the top three teams in Challenge 1 of the GO Competition (Teams gollnlp, GO-SNIP, and GMI-GO).
△ Less
Submitted 15 June, 2022;
originally announced June 2022.
-
Derivative-Free Bound-Constrained Optimization for Solving Structured Problems with Surrogate Models
Authors:
Frank E. Curtis,
Shima Dezfulian,
Andreas Wächter
Abstract:
We propose and analyze a model-based derivative-free (DFO) algorithm for solving bound-constrained optimization problems where the objective function is the composition of a smooth function and a vector of black-box functions. We assume that the black-box functions are smooth and the evaluation of them is the computational bottleneck of the algorithm. The distinguishing feature of our algorithm is…
▽ More
We propose and analyze a model-based derivative-free (DFO) algorithm for solving bound-constrained optimization problems where the objective function is the composition of a smooth function and a vector of black-box functions. We assume that the black-box functions are smooth and the evaluation of them is the computational bottleneck of the algorithm. The distinguishing feature of our algorithm is the use of approximate function values at interpolation points which can be obtained by an application-specific surrogate model that is cheap to evaluate. As an example, we consider the situation in which a sequence of related optimization problems is solved and present a regression-based approximation scheme that uses function values that were evaluated when solving prior problem instances. In addition, we propose and analyze a new algorithm for obtaining interpolation points that handles unrelaxable bound constraints. Our numerical results show that our algorithm outperforms a state-of-the-art DFO algorithm for solving a least-squares problem from a chemical engineering application when a history of black-box function evaluations is available.
△ Less
Submitted 2 January, 2024; v1 submitted 25 February, 2022;
originally announced February 2022.
-
A Versatile Optimization Framework For Sustainable Post-Disaster Building Reconstruction
Authors:
Niloufar Izadinia,
Elham Ramyar,
Maytham Alzayer,
Stephen H. Carr,
Gianluca Cusatis,
Vidushi Dwivedi,
Daniel J. Garcia,
Missaka Hettiarachchi,
Thomas Massion,
William M. Miller,
Andreas Wächter
Abstract:
This paper proposes an optimization framework for sustainable post-disaster building reconstruction. Based on mathematical optimization, it is intended to provide decision-makers with a versatile tool to optimize building designs and to explore the trade-off between costs and environmental impact (represented by embodied energy) of alternative building materials. The mixed-integer nonlinear optimi…
▽ More
This paper proposes an optimization framework for sustainable post-disaster building reconstruction. Based on mathematical optimization, it is intended to provide decision-makers with a versatile tool to optimize building designs and to explore the trade-off between costs and environmental impact (represented by embodied energy) of alternative building materials. The mixed-integer nonlinear optimization model includes an analytical building model that considers structural and safety constraints and incorporates regional building codes. Using multi-objective optimization concepts, Pareto-optimal designs are computed that represent the best trade-off designs from which a decision-maker can choose when they take additional criteria into consideration. As a case study, we consider the design of a multi-room one-story masonry building in Nepal. We demonstrate how the framework can be employed to answer a variety of questions, such as the optimal building design and material selection, the sensitivity of the decision to the prices, and the impact of regional safety regulation thresholds.
△ Less
Submitted 18 August, 2022; v1 submitted 25 November, 2021;
originally announced November 2021.
-
A Decomposition Algorithm for Large-Scale Security-Constrained AC Optimal Power Flow
Authors:
Frank E. Curtis,
Daniel K. Molzahn,
Shenyinying Tu,
Andreas Wächter,
Ermin Wei,
Elizabeth Wong
Abstract:
A decomposition algorithm for solving large-scale security-constrained AC optimal power flow problems is presented. The formulation considered is the one used in the ARPA-E Grid Optimization (GO) Competition, Challenge 1, held from November 2018 through October 2019. The techniques found to be most effective in terms of performance in the challenge are presented, including strategies for contingen…
▽ More
A decomposition algorithm for solving large-scale security-constrained AC optimal power flow problems is presented. The formulation considered is the one used in the ARPA-E Grid Optimization (GO) Competition, Challenge 1, held from November 2018 through October 2019. The techniques found to be most effective in terms of performance in the challenge are presented, including strategies for contingency selection, fast contingency evaluation, handling complementarity constraints, avoiding issues related to degeneracy, and exploiting parallelism. The results of numerical experiments are provided to demonstrate the effectiveness of the proposed techniques as compared to alternative strategies.
△ Less
Submitted 4 October, 2021;
originally announced October 2021.
-
A Two-Stage Decomposition Approach for AC Optimal Power Flow
Authors:
Shenyinying Tu,
Andreas Waechter,
Ermin Wei
Abstract:
The alternating current optimal power flow (AC-OPF) problem is critical to power system operations and planning, but it is generally hard to solve due to its nonconvex and large-scale nature. This paper proposes a scalable decomposition approach in which the power network is decomposed into a master network and a number of subnetworks, where each network has its own AC-OPF subproblem. This formula…
▽ More
The alternating current optimal power flow (AC-OPF) problem is critical to power system operations and planning, but it is generally hard to solve due to its nonconvex and large-scale nature. This paper proposes a scalable decomposition approach in which the power network is decomposed into a master network and a number of subnetworks, where each network has its own AC-OPF subproblem. This formulates a two-stage optimization problem and requires only a small amount of communication between the master and subnetworks. The key contribution is a smoothing technique that renders the response of a subnetwork differentiable with respect to the input from the master problem, utilizing properties of the barrier problem formulation that naturally arises when subproblems are solved by a primal-dual interior-point algorithm. Consequently, existing efficient nonlinear programming solvers can be used for both the master problem and the subproblems. The advantage of this framework is that speedup can be obtained by processing the subnetworks in parallel, and it has convergence guarantees under reasonable assumptions. The formulation is readily extended to instances with stochastic subnetwork loads. Numerical results show favorable performance and illustrate the scalability of the algorithm which is able to solve instances with more than 11 million buses.
△ Less
Submitted 10 June, 2020; v1 submitted 18 February, 2020;
originally announced February 2020.
-
DC Optimal Power Flow with Joint Chance Constraints
Authors:
Alejandra Pena-Ordieres,
Daniel Molzahn,
Line Roald,
Andreas Waechter
Abstract:
Managing uncertainty and variability in power injections has become a major concern for power system operators due to the increasing levels of fluctuating renewable energy connected to the grid. This work addresses this uncertainty via a joint chance-constrained formulation of the DC optimal power flow (OPF) problem, which satisfies \emph{all} the constraints \emph{jointly} with a pre-determined p…
▽ More
Managing uncertainty and variability in power injections has become a major concern for power system operators due to the increasing levels of fluctuating renewable energy connected to the grid. This work addresses this uncertainty via a joint chance-constrained formulation of the DC optimal power flow (OPF) problem, which satisfies \emph{all} the constraints \emph{jointly} with a pre-determined probability. The few existing approaches for solving joint chance-constrained OPF problems are typically either computationally intractable for large-scale problems or give overly conservative solutions that satisfy the constraints far more often than required, resulting in excessively costly operation. This paper proposes an algorithm for solving joint chance-constrained DC OPF problems by adopting an S$\ell_1$QP-type trust-region algorithm. This algorithm uses a sample-based approach that avoids making strong assumptions on the distribution of the uncertainties, scales favorably to large problems, and can be tuned to obtain less conservative results. We illustrate the performance of our method using several IEEE test cases. The results demonstrate the proposed algorithm's advantages in computational times and limited conservativeness of the solutions relative to other joint chance-constrained DC OPF algorithms.
△ Less
Submitted 2 May, 2020; v1 submitted 27 November, 2019;
originally announced November 2019.
-
Exploring Benefits of Linear Solver Parallelism on Modern Nonlinear Optimization Applications
Authors:
Byron Tasseff,
Carleton Coffrin,
Andreas Wächter,
Carl Laird
Abstract:
The advent of efficient interior point optimization methods has enabled the tractable solution of large-scale linear and nonlinear programming (NLP) problems. A prominent example of such a method is seen in Ipopt, a widely-used, open-source nonlinear optimization solver. Algorithmically, Ipopt depends on the use of a sparse symmetric indefinite linear system solver, which is heavily employed withi…
▽ More
The advent of efficient interior point optimization methods has enabled the tractable solution of large-scale linear and nonlinear programming (NLP) problems. A prominent example of such a method is seen in Ipopt, a widely-used, open-source nonlinear optimization solver. Algorithmically, Ipopt depends on the use of a sparse symmetric indefinite linear system solver, which is heavily employed within the optimization of barrier subproblems. As such, the performance and reliability of Ipopt is dependent on the properties of the selected linear solver. Inspired by a trend in mathematical programming toward solving larger and more challenging NLPs, this work explores two core questions: first, how does the scalability of available linear solvers, many of which exhibit shared-memory parallelism, impact Ipopt performance; and second, does the best linear solver vary across NLP problem classes, including nonlinear network problems and problems constrained by partial differential equations? To better understand these properties, this paper first describes available open- and closed-source, serial and parallel linear solvers and the fundamental differences among them. Second, it introduces the coupling of a new open-source linear solver capable of heterogeneous parallelism over multi-core central processing units and graphics processing units. Third, it compares linear solvers using a variety of mathematical programming problems, including standard test problems for linear and nonlinear optimization, optimal power flow benchmarks, and scalable two- and three-dimensional partial differential equation and optimal control problems. Finally, linear solver recommendations are provided to maximize Ipopt performance across different application domains.
△ Less
Submitted 17 September, 2019;
originally announced September 2019.
-
Solving Chance-Constrained Problems via a Smooth Sample-Based Nonlinear Approximation
Authors:
Alejandra Peña-Ordieres,
James R. Luedtke,
Andreas Wächter
Abstract:
We introduce a new method for solving nonlinear continuous optimization problems with chance constraints. Our method is based on a reformulation of the probabilistic constraint as a quantile function. The quantile function is approximated via a differentiable sample average approximation. We provide theoretical statistical guarantees of the approximation, and illustrate empirically that the reform…
▽ More
We introduce a new method for solving nonlinear continuous optimization problems with chance constraints. Our method is based on a reformulation of the probabilistic constraint as a quantile function. The quantile function is approximated via a differentiable sample average approximation. We provide theoretical statistical guarantees of the approximation, and illustrate empirically that the reformulation can be directly used by standard nonlinear optimization solvers in the case of single chance constraints. Furthermore, we propose an S$\ell_1$QP-type trust-region method to solve instances with joint chance constraints. We demonstrate the performance of the method on several problems, and show that it scales well with the sample size and that the smoothing can be used to counteract the bias in the chance constraint approximation induced by the sample approximation.
△ Less
Submitted 14 March, 2020; v1 submitted 17 May, 2019;
originally announced May 2019.
-
On the convergence of BFGS on a class of piecewise linear non-smooth functions
Authors:
Yuchen Xie,
Andreas Waechter
Abstract:
The quasi-Newton Broyden-Fletcher-Goldfarb-Shanno (BFGS) method has proven to be very reliable and efficient for the minimization of smooth objective functions since its inception in the 1960s. Recently, it was observed empirically that it also works remarkably well for non-smooth problems when combined with the Armijo-Wolfe line search, but only very limited theoretical convergence theory could b…
▽ More
The quasi-Newton Broyden-Fletcher-Goldfarb-Shanno (BFGS) method has proven to be very reliable and efficient for the minimization of smooth objective functions since its inception in the 1960s. Recently, it was observed empirically that it also works remarkably well for non-smooth problems when combined with the Armijo-Wolfe line search, but only very limited theoretical convergence theory could be established so far. In this paper, we expand these results by considering convex piecewise linear functions with two pieces that are not bounded below. We prove that the algorithm always terminates in a finite number of iterations, eventually generating an unbounded direction. In other words, in contrast to the gradient method, the BFGS algorithm does not converge to a non-stationary point.
△ Less
Submitted 22 December, 2017;
originally announced December 2017.
-
A Limited-Memory Quasi-Newton Algorithm for Bound-Constrained Nonsmooth Optimization
Authors:
Nitish Shirish Keskar,
Andreas Waechter
Abstract:
We consider the problem of minimizing a continuous function that may be nonsmooth and nonconvex, subject to bound constraints. We propose an algorithm that uses the L-BFGS quasi-Newton approximation of the problem's curvature together with a variant of the weak Wolfe line search. The key ingredient of the method is an active-set selection strategy that defines the subspace in which search directio…
▽ More
We consider the problem of minimizing a continuous function that may be nonsmooth and nonconvex, subject to bound constraints. We propose an algorithm that uses the L-BFGS quasi-Newton approximation of the problem's curvature together with a variant of the weak Wolfe line search. The key ingredient of the method is an active-set selection strategy that defines the subspace in which search directions are computed. To overcome the inherent shortsightedness of the gradient for a nonsmooth function, we propose two strategies. The first relies on an approximation of the $ε$-minimum norm subgradient, and the second uses an iterative corrective loop that augments the active set based on the resulting search directions. We describe a Python implementation of the proposed algorithm and present numerical results on a set of standard test problems to illustrate the efficacy of our approach.
△ Less
Submitted 21 December, 2016;
originally announced December 2016.
-
A Second-Order Method for Convex $\ell_1$-Regularized Optimization with Active Set Prediction
Authors:
Nitish Shirish Keskar,
Jorge Nocedal,
Figen Oztoprak,
Andreas Waechter
Abstract:
We describe an active-set method for the minimization of an objective function $φ$ that is the sum of a smooth convex function and an $\ell_1$-regularization term. A distinctive feature of the method is the way in which active-set identification and {second-order} subspace minimization steps are integrated to combine the predictive power of the two approaches. At every iteration, the algorithm sel…
▽ More
We describe an active-set method for the minimization of an objective function $φ$ that is the sum of a smooth convex function and an $\ell_1$-regularization term. A distinctive feature of the method is the way in which active-set identification and {second-order} subspace minimization steps are integrated to combine the predictive power of the two approaches. At every iteration, the algorithm selects a candidate set of free and fixed variables, performs an (inexact) subspace phase, and then assesses the quality of the new active set. If it is not judged to be acceptable, then the set of free variables is restricted and a new active-set prediction is made. We establish global convergence for our approach, and compare the new method against the state-of-the-art code LIBLINEAR.
△ Less
Submitted 16 May, 2015;
originally announced May 2015.
