-
Polynomial worst-case iteration complexity of quasi-Newton primal-dual interior point algorithms for linear programming
Authors:
Jacek Gondzio,
Francisco N. C. Sobral
Abstract:
Quasi-Newton methods are well known techniques for large-scale numerical optimization. They use an approximation of the Hessian in optimization problems or the Jacobian in system of nonlinear equations. In the Interior Point context, quasi-Newton algorithms compute low-rank updates of the matrix associated with the Newton systems, instead of computing it from scratch at every iteration. In this wo…
▽ More
Quasi-Newton methods are well known techniques for large-scale numerical optimization. They use an approximation of the Hessian in optimization problems or the Jacobian in system of nonlinear equations. In the Interior Point context, quasi-Newton algorithms compute low-rank updates of the matrix associated with the Newton systems, instead of computing it from scratch at every iteration. In this work, we show that a simplified quasi-Newton primal-dual interior point algorithm for linear programming enjoys polynomial worst-case iteration complexity. Feasible and infeasible cases of the algorithm are considered and the most common neighborhoods of the central path are analyzed. To the best of our knowledge, this is the first attempt to deliver polynomial worst-case iteration complexity bounds for these methods. Unsurprisingly, the worst-case complexity results obtained when quasi-Newton directions are used are worse than their counterparts when Newton directions are employed. However, quasi-Newton updates are very attractive for large-scale optimization problems where the cost of factorizing the matrices is much higher than the cost of solving linear systems.
△ Less
Submitted 9 September, 2022; v1 submitted 18 August, 2022;
originally announced August 2022.
-
On complexity constants of linear and quadratic models for derivative-free trust-region algorithms
Authors:
A. E. Schwertner,
F. N. C. Sobral
Abstract:
Complexity analysis has become an important tool in the convergence analysis of optimization algorithms. For derivative-free optimization algorithms, it is not different. Interestingly, several constants that appear when developing complexity results hide the dimensions of the problem. This work organizes several results in literature about bounds that appear in derivative-free trust-region algori…
▽ More
Complexity analysis has become an important tool in the convergence analysis of optimization algorithms. For derivative-free optimization algorithms, it is not different. Interestingly, several constants that appear when developing complexity results hide the dimensions of the problem. This work organizes several results in literature about bounds that appear in derivative-free trust-region algorithms based on linear and quadratic models. All the constants are given explicitly by the quality of the sample set, dimension of the problem and number of sample points. We extend some results to allow "inexact" interpolation sets. We also provide a clearer proof than those already existing in literature for the underdetermined case.
△ Less
Submitted 24 September, 2024; v1 submitted 23 May, 2022;
originally announced May 2022.
-
A robust method based on LOVO functions for solving least squares problems
Authors:
E. V. Castelani,
R. Lopes,
W. V. I. Shirabayashi,
F. N. C. Sobral
Abstract:
The robust adjustment of nonlinear models to data is considered in this paper. When data comes from real experiments, it is possible that measurement errors cause the appearance of discrepant values, which should be ignored when adjusting models to them. This work presents a Lower Order-value Optimization (LOVO) version of the Levenberg-Marquardt algorithm, which is well suited to deal with outlie…
▽ More
The robust adjustment of nonlinear models to data is considered in this paper. When data comes from real experiments, it is possible that measurement errors cause the appearance of discrepant values, which should be ignored when adjusting models to them. This work presents a Lower Order-value Optimization (LOVO) version of the Levenberg-Marquardt algorithm, which is well suited to deal with outliers in fitting problems. A general algorithm is presented and convergence to stationary points is demonstrated. Numerical results show that the algorithm is successfully able to detect and ignore outliers without too many specific parameters. Parallel and distributed executions of the algorithm are also possible, allowing for the use of larger datasets. Comparison against publicly available robust algorithms shows that the present approach is able to find better adjustments in well known statistical models.
△ Less
Submitted 29 November, 2019;
originally announced November 2019.
-
Quasi-Newton approaches to Interior Point Methods for quadratic problems
Authors:
J. Gondzio,
F. N. C. Sobral
Abstract:
Interior Point Methods (IPM) rely on the Newton method for solving systems of nonlinear equations. Solving the linear systems which arise from this approach is the most computationally expensive task of an interior point iteration. If, due to problem's inner structure, there are special techniques for efficiently solving linear systems, IPMs enjoy fast convergence and are able to solve large scale…
▽ More
Interior Point Methods (IPM) rely on the Newton method for solving systems of nonlinear equations. Solving the linear systems which arise from this approach is the most computationally expensive task of an interior point iteration. If, due to problem's inner structure, there are special techniques for efficiently solving linear systems, IPMs enjoy fast convergence and are able to solve large scale optimization problems. It is tempting to try to replace the Newton method by quasi-Newton methods. Quasi-Newton approaches to IPMs either are built to approximate the Lagrangian function for nonlinear programming problems or provide an inexpensive preconditioner. In this work we study the impact of using quasi-Newton methods applied directly to the nonlinear system of equations for general quadratic programming problems. The cost of each iteration can be compared to the cost of computing correctors in a usual interior point iteration. Numerical experiments show that the new approach is able to reduce the overall number of matrix factorizations and is suitable for a matrix-free implementation.
△ Less
Submitted 26 June, 2018;
originally announced June 2018.
