Showing 1–36 of 36 results for author: Gondzio, J

An efficient active-set method with applications to sparse approximations and risk minimization

Authors: Spyridon Pougkakiotis, Jacek Gondzio, Dionysis Kalogerias

Abstract: In this paper we present an efficient active-set method for the solution of convex quadratic programming problems with general piecewise-linear terms in the objective, with applications to sparse approximations and risk-minimization. The algorithm is derived by combining a proximal method of multipliers (PMM) with a standard semismooth Newton method (SSN), and is shown to be globally convergent un… ▽ More In this paper we present an efficient active-set method for the solution of convex quadratic programming problems with general piecewise-linear terms in the objective, with applications to sparse approximations and risk-minimization. The algorithm is derived by combining a proximal method of multipliers (PMM) with a standard semismooth Newton method (SSN), and is shown to be globally convergent under minimal assumptions. Further local linear (and potentially superlinear) convergence is shown under standard additional conditions. The major computational bottleneck of the proposed approach arises from the solution of the associated SSN linear systems. These are solved using a Krylov-subspace method, accelerated by certain novel general-purpose preconditioners which are shown to be optimal with respect to the proximal penalty parameters. The preconditioners are easy to store and invert, since they exploit the structure of the nonsmooth terms appearing in the problem's objective to significantly reduce their memory requirements. We showcase the efficiency, robustness, and scalability of the proposed solver on a variety of problems arising in risk-averse portfolio selection, $L^1$-regularized partial differential equation constrained optimization, quantile regression, and binary classification via linear support vector machines. We provide computational evidence, on real-world datasets, to demonstrate the ability of the solver to efficiently and competitively handle a diverse set of medium- and large-scale optimization instances. △ Less

Submitted 7 May, 2024; originally announced May 2024.

Comments: arXiv admin note: substantial text overlap with arXiv:2302.14497, arXiv:2201.10211

A regularized Interior Point Method for sparse Optimal Transport on Graphs

Authors: Stefano Cipolla, Jacek Gondzio, Filippo Zanetti

Abstract: In this work, the authors address the Optimal Transport (OT) problem on graphs using a proximal stabilized Interior Point Method (IPM). In particular, strongly leveraging on the induced primal-dual regularization, the authors propose to solve large scale OT problems on sparse graphs using a bespoke IPM algorithm able to suitably exploit primal-dual regularization in order to enforce scalability. I… ▽ More In this work, the authors address the Optimal Transport (OT) problem on graphs using a proximal stabilized Interior Point Method (IPM). In particular, strongly leveraging on the induced primal-dual regularization, the authors propose to solve large scale OT problems on sparse graphs using a bespoke IPM algorithm able to suitably exploit primal-dual regularization in order to enforce scalability. Indeed, the authors prove that the introduction of the regularization allows to use sparsified versions of the normal Newton equations to inexpensively generate IPM search directions. A detailed theoretical analysis is carried out showing the polynomial convergence of the inner algorithm in the proposed computational framework. Moreover, the presented numerical results showcase the efficiency and robustness of the proposed approach when compared to network simplex solvers. △ Less

Submitted 11 July, 2023; originally announced July 2023.

An active-set method for sparse approximations. Part II: General piecewise-linear terms

Authors: Spyridon Pougkakiotis, Jacek Gondzio, Dionysios S. Kalogerias

Abstract: In this paper we present an efficient active-set method for the solution of convex quadratic programming problems with general piecewise-linear terms in the objective, with applications to sparse approximations and risk-minimization. The method exploits the structure of the piecewise-linear terms appearing in the objective in order to significantly reduce its memory requirements, and thus improve… ▽ More In this paper we present an efficient active-set method for the solution of convex quadratic programming problems with general piecewise-linear terms in the objective, with applications to sparse approximations and risk-minimization. The method exploits the structure of the piecewise-linear terms appearing in the objective in order to significantly reduce its memory requirements, and thus improve its efficiency. We showcase the robustness of the proposed solver on a variety of problems arising in risk-averse portfolio selection, quantile regression, and binary classification via linear support vector machines. We provide computational evidence to demonstrate, on real-world datasets, the ability of the solver of efficiently handling a variety of problems, by comparing it against an efficient general-purpose interior point solver as well as a state-of-the-art alternating direction method of multipliers. This work complements the accompanying paper [``An active-set method for sparse approximations. Part I: Separable $\ell_1$ terms", S. Pougkakiotis, J. Gondzio, D. S. Kalogerias], in which we discuss the case of separable $\ell_1$ terms, analyze the convergence, and propose general-purpose preconditioning strategies for the solution of its associated linear systems. △ Less

Submitted 28 February, 2023; originally announced February 2023.

Polynomial worst-case iteration complexity of quasi-Newton primal-dual interior point algorithms for linear programming

Authors: Jacek Gondzio, Francisco N. C. Sobral

Abstract: Quasi-Newton methods are well known techniques for large-scale numerical optimization. They use an approximation of the Hessian in optimization problems or the Jacobian in system of nonlinear equations. In the Interior Point context, quasi-Newton algorithms compute low-rank updates of the matrix associated with the Newton systems, instead of computing it from scratch at every iteration. In this wo… ▽ More Quasi-Newton methods are well known techniques for large-scale numerical optimization. They use an approximation of the Hessian in optimization problems or the Jacobian in system of nonlinear equations. In the Interior Point context, quasi-Newton algorithms compute low-rank updates of the matrix associated with the Newton systems, instead of computing it from scratch at every iteration. In this work, we show that a simplified quasi-Newton primal-dual interior point algorithm for linear programming enjoys polynomial worst-case iteration complexity. Feasible and infeasible cases of the algorithm are considered and the most common neighborhoods of the central path are analyzed. To the best of our knowledge, this is the first attempt to deliver polynomial worst-case iteration complexity bounds for these methods. Unsurprisingly, the worst-case complexity results obtained when quasi-Newton directions are used are worse than their counterparts when Newton directions are employed. However, quasi-Newton updates are very attractive for large-scale optimization problems where the cost of factorizing the matrices is much higher than the cost of solving linear systems. △ Less

Submitted 9 September, 2022; v1 submitted 18 August, 2022; originally announced August 2022.

MSC Class: 90C05 (Secondary); 90C51 (Primary); 90C53

An Interior-Point-Inspired algorithm for Linear Programs arising in Discrete Optimal Transport

Authors: Filippo Zanetti, Jacek Gondzio

Abstract: Discrete Optimal Transport problems give rise to very large linear programs (LP) with a particular structure of the constraint matrix. In this paper we present a hybrid algorithm that mixes an interior point method (IPM) and column generation, specialized for the LP originating from the Kantorovich Optimal Transport problem. Knowing that optimal solutions of such problems display a high degree of… ▽ More Discrete Optimal Transport problems give rise to very large linear programs (LP) with a particular structure of the constraint matrix. In this paper we present a hybrid algorithm that mixes an interior point method (IPM) and column generation, specialized for the LP originating from the Kantorovich Optimal Transport problem. Knowing that optimal solutions of such problems display a high degree of sparsity, we propose a column-generation-like technique to force all intermediate iterates to be as sparse as possible. The algorithm is implemented nearly matrix-free. Indeed, most of the computations avoid forming the huge matrices involved and solve the Newton system using only a much smaller Schur complement of the normal equations. We prove theoretical results about the sparsity pattern of the optimal solution, exploiting the graph structure of the underlying problem. We use these results to mix iterative and direct linear solvers efficiently, in a way that avoids producing preconditioners or factorizations with excessive fill-in and at the same time guaranteeing a low number of conjugate gradient iterations. We compare the proposed method with two state-of-the-art solvers and show that it can compete with the best network optimization tools in terms of computational time and memory usage. We perform experiments with problems reaching more than four billion variables and demonstrate the robustness of the proposed method. △ Less

Submitted 9 December, 2022; v1 submitted 22 June, 2022; originally announced June 2022.

Proximal stabilized Interior Point Methods for quadratic programming and low-frequency-updates preconditioning techniques

Authors: Stefano Cipolla, Jacek Gondzio

Abstract: In this work, in the context of Linear and Quadratic Programming, we interpret Primal Dual Regularized Interior Point Methods (PDR-IPMs) in the framework of the Proximal Point Method. The resulting Proximal Stabilized IPM (PS-IPM) is strongly supported by theoretical results concerning convergence and the rate of convergence, and can handle degenerate problems. Moreover, in the second part of this… ▽ More In this work, in the context of Linear and Quadratic Programming, we interpret Primal Dual Regularized Interior Point Methods (PDR-IPMs) in the framework of the Proximal Point Method. The resulting Proximal Stabilized IPM (PS-IPM) is strongly supported by theoretical results concerning convergence and the rate of convergence, and can handle degenerate problems. Moreover, in the second part of this work, we analyse the interactions between the regularization parameters and the computational foot-print of the linear algebra routines used to solve the Newton linear systems. In particular, when these systems are solved using an iterative Krylov method, we propose general purpose preconditioners which, exploiting the regularization and a new rearrangement of the Schur complement, remain attractive for a series of subsequent IPM iterations. Therefore they need to be recomputed only in a fraction of the total IPM iterations. The resulting regularized second order methods, for which low-frequency-updates of the preconditioners are allowed, pave the path for an alternative third way in-between first and second order methods. △ Less

Submitted 3 May, 2022; originally announced May 2022.

An active-set method for sparse approximations. Part I: Separable $\ell_1$ terms

Authors: Spyridon Pougkakiotis, Jacek Gondzio, Dionysios S. Kalogerias

Abstract: In this paper we present an active-set method for the solution of $\ell_1$-regularized convex quadratic optimization problems. It is derived by combining a proximal method of multipliers (PMM) strategy with a standard semismooth Newton method (SSN). The resulting linear systems are solved using a Krylov-subspace method, accelerated by certain general-purpose preconditioners which are shown to be o… ▽ More In this paper we present an active-set method for the solution of $\ell_1$-regularized convex quadratic optimization problems. It is derived by combining a proximal method of multipliers (PMM) strategy with a standard semismooth Newton method (SSN). The resulting linear systems are solved using a Krylov-subspace method, accelerated by certain general-purpose preconditioners which are shown to be optimal with respect to the proximal parameters. Practical efficiency is further improved by warm-starting the algorithm using a proximal alternating direction method of multipliers. We show that the outer PMM achieves global convergence under mere feasibility assumptions. Under additional standard assumptions, the PMM scheme achieves global linear and local superlinear convergence. The SSN scheme is locally superlinearly convergent, assuming that its associated linear systems are solved accurately enough, and globally convergent under certain additional regularity assumptions. We provide numerical evidence to demonstrate the effectiveness of the approach by comparing it against OSQP and IP-PMM (an ADMM and a regularized IPM solver, respectively) on several elastic-net linear regression and $L^1$-regularized PDE-constrained optimization problems. △ Less

Submitted 28 February, 2023; v1 submitted 25 January, 2022; originally announced January 2022.

Training very large scale nonlinear SVMs using Alternating Direction Method of Multipliers coupled with the Hierarchically Semi-Separable kernel approximations

Authors: S. Cipolla, J. Gondzio

Abstract: Typically, nonlinear Support Vector Machines (SVMs) produce significantly higher classification quality when compared to linear ones but, at the same time, their computational complexity is prohibitive for large-scale datasets: this drawback is essentially related to the necessity to store and manipulate large, dense and unstructured kernel matrices. Despite the fact that at the core of training a… ▽ More Typically, nonlinear Support Vector Machines (SVMs) produce significantly higher classification quality when compared to linear ones but, at the same time, their computational complexity is prohibitive for large-scale datasets: this drawback is essentially related to the necessity to store and manipulate large, dense and unstructured kernel matrices. Despite the fact that at the core of training a SVM there is a \textit{simple} convex optimization problem, the presence of kernel matrices is responsible for dramatic performance reduction, making SVMs unworkably slow for large problems. Aiming to an efficient solution of large-scale nonlinear SVM problems, we propose the use of the \textit{Alternating Direction Method of Multipliers} coupled with \textit{Hierarchically Semi-Separable} (HSS) kernel approximations. As shown in this work, the detailed analysis of the interaction among their algorithmic components unveils a particularly efficient framework and indeed, the presented experimental results demonstrate a significant speed-up when compared to the \textit{state-of-the-art} nonlinear SVM libraries (without significantly affecting the classification accuracy). △ Less

Submitted 10 November, 2021; v1 submitted 9 August, 2021; originally announced August 2021.

General-purpose preconditioning for regularized interior point methods

Authors: Jacek Gondzio, Spyridon Pougkakiotis, John W. Pearson

Abstract: In this paper we present general-purpose preconditioners for regularized augmented systems arising from optimization problems, and their corresponding normal equations. We discuss positive definite preconditioners, suitable for CG and MINRES. We consider "sparsifications" which avoid situations in which eigenvalues of the preconditioned matrix may become complex. Special attention is given to syst… ▽ More In this paper we present general-purpose preconditioners for regularized augmented systems arising from optimization problems, and their corresponding normal equations. We discuss positive definite preconditioners, suitable for CG and MINRES. We consider "sparsifications" which avoid situations in which eigenvalues of the preconditioned matrix may become complex. Special attention is given to systems arising from the application of regularized interior point methods to linear or nonlinear convex programming problems. △ Less

Submitted 31 March, 2022; v1 submitted 14 July, 2021; originally announced July 2021.

Material-separating regularizer for multi-energy X-ray tomography

Authors: Jacek Gondzio, Matti Lassas, Salla-Maaria Latva-Äijö, Samuli Siltanen, Filippo Zanetti

Abstract: Dual-energy X-ray tomography is considered in a context where the target under imaging consists of two distinct materials. The materials are assumed to be possibly intertwined in space, but at any given location there is only one material present. Further, two X-ray energies are chosen so that there is a clear difference in the spectral dependence of the attenuation coefficients of the two materia… ▽ More Dual-energy X-ray tomography is considered in a context where the target under imaging consists of two distinct materials. The materials are assumed to be possibly intertwined in space, but at any given location there is only one material present. Further, two X-ray energies are chosen so that there is a clear difference in the spectral dependence of the attenuation coefficients of the two materials. A novel regularizer is presented for the inverse problem of reconstructing separate tomographic images for the two materials. A combination of two things, (a) non-negativity constraint, and (b) penalty term containing the inner product between the two material images, promotes the presence of at most one material in a given pixel. A preconditioned interior point method is derived for the minimization of the regularization functional. Numerical tests with digital phantoms suggest that the new algorithm outperforms the baseline method, Joint Total Variation regularization, in terms of correctly material-characterized pixels. While the method is tested only in a two-dimensional setting with two materials and two energies, the approach readily generalizes to three dimensions and more materials. The number of materials just needs to match the number of energies used in imaging. △ Less

Submitted 28 June, 2021; originally announced July 2021.

Comments: 31 pages, 11 figures, 4 tables

Journal ref: Inverse Problems 38.2 (2022): 025013

A new stopping criterion for Krylov solvers applied in Interior Point Methods

Authors: Filippo Zanetti, Jacek Gondzio

Abstract: When an iterative method is applied to solve the linear equation system in interior point methods (IPMs), the attention is usually placed on accelerating their convergence by designing appropriate preconditioners, but the linear solver is applied as a black box with a standard termination criterion which asks for a sufficient reduction of the residual in the linear system. Such an approach often l… ▽ More When an iterative method is applied to solve the linear equation system in interior point methods (IPMs), the attention is usually placed on accelerating their convergence by designing appropriate preconditioners, but the linear solver is applied as a black box with a standard termination criterion which asks for a sufficient reduction of the residual in the linear system. Such an approach often leads to an unnecessary 'oversolving' of linear equations. In this paper, an IPM that relies on an inner termination criterion not based on the residual of the linear system is introduced and analyzed. Moreover, new indicators for the early termination of the inner iterations are derived from a deep understanding of IPM needs. The new technique has been adapted to the Conjugate Gradient (CG) and to the Minimum Residual method (MINRES) applied in the IPM context. The new criterion has been tested on a set of quadratic optimization problems including compressed sensing, image processing and instances with partial differential equation constraints, and it has been compared to standard residual tests with variable tolerance. Evidence gathered from these computational experiments shows that the new technique delivers significant improvements in terms of inner (linear) iterations and those translate into significant savings of the IPM solution time. △ Less

Submitted 19 September, 2022; v1 submitted 30 June, 2021; originally announced June 2021.

Journal ref: SIAM J Sci Comput, 45(2), 2023, A703-A728

Sparse Approximations with Interior Point Methods

Authors: Valentina De Simone, Daniela di Serafino, Jacek Gondzio, Spyridon Pougkakiotis, Marco Viola

Abstract: Large-scale optimization problems that seek sparse solutions have become ubiquitous. They are routinely solved with various specialized first-order methods. Although such methods are often fast, they usually struggle with not-so-well conditioned problems. In this paper, specialized variants of an interior point-proximal method of multipliers are proposed and analyzed for problems of this class. Co… ▽ More Large-scale optimization problems that seek sparse solutions have become ubiquitous. They are routinely solved with various specialized first-order methods. Although such methods are often fast, they usually struggle with not-so-well conditioned problems. In this paper, specialized variants of an interior point-proximal method of multipliers are proposed and analyzed for problems of this class. Computational experience on a variety of problems, namely, multi-period portfolio optimization, classification of data coming from functional Magnetic Resonance Imaging, restoration of images corrupted by Poisson noise, and classification via regularized logistic regression, provides substantial evidence that interior point methods, equipped with suitable linear algebra, can offer a noticeable advantage over first-order approaches. △ Less

Submitted 25 November, 2021; v1 submitted 26 February, 2021; originally announced February 2021.

Comments: 34 pages, 5 figures, 5 tables

MSC Class: 65K05; 90C51; 90C25; 65F10; 65F08; 90C90

Random multi-block ADMM: an ALM based view for the QP case

Authors: Stefano Cipolla, Jacek Gondzio

Abstract: Embedding randomization procedures in the Alternating Direction Method of Multipliers (ADMM) has recently attracted an increasing amount of interest as a remedy to the fact that the direct multi-block generalization of ADMM is not necessarily convergent. Even if, in practice, the introduction of such techniques could \textit{mitigate} the diverging behaviour of the multi-block extension of ADMM, f… ▽ More Embedding randomization procedures in the Alternating Direction Method of Multipliers (ADMM) has recently attracted an increasing amount of interest as a remedy to the fact that the direct multi-block generalization of ADMM is not necessarily convergent. Even if, in practice, the introduction of such techniques could \textit{mitigate} the diverging behaviour of the multi-block extension of ADMM, from the theoretical point of view, it can ensure just the \textit{convergence in expectation}, which may not be a good indicator of its robustness and efficiency. In this work, analysing the strongly convex quadratic programming case, we interpret the block Gauss-Seidel sweep performed by the multi-block ADMM in the context of the inexact Augmented Lagrangian Method. Using the proposed analysis, we are able to outline an alternative technique to those present in literature which, supported from stronger theoretical guarantees, is able to ensure the convergence of the multi-block generalization of the ADMM method. △ Less

Submitted 8 July, 2021; v1 submitted 16 December, 2020; originally announced December 2020.

MSC Class: 90C25; 65K05; 65F10

An Interior Point-Proximal Method of Multipliers for Positive Semi-Definite Programming

Authors: Spyridon Pougkakiotis, Jacek Gondzio

Abstract: In this paper we generalize the Interior Point-Proximal Method of Multipliers (IP-PMM) presented in [An Interior Point-Proximal Method of Multipliers for Convex Quadratic Programming, Computational Optimization and Applications, 78, 307--351 (2021)] for the solution of linear positive Semi-Definite Programming (SDP) problems, allowing inexactness in the solution of the associated Newton systems. I… ▽ More In this paper we generalize the Interior Point-Proximal Method of Multipliers (IP-PMM) presented in [An Interior Point-Proximal Method of Multipliers for Convex Quadratic Programming, Computational Optimization and Applications, 78, 307--351 (2021)] for the solution of linear positive Semi-Definite Programming (SDP) problems, allowing inexactness in the solution of the associated Newton systems. In particular, we combine an infeasible Interior Point Method (IPM) with the Proximal Method of Multipliers (PMM) and interpret the algorithm (IP-PMM) as a primal-dual regularized IPM, suitable for solving SDP problems. We apply some iterations of an IPM to each sub-problem of the PMM until a satisfactory solution is found. We then update the PMM parameters, form a new IPM neighbourhood, and repeat this process. Given this framework, we prove polynomial complexity of the algorithm, under mild assumptions, and without requiring exact computations for the Newton directions. We furthermore provide a necessary condition for lack of strong duality, which can be used as a basis for constructing detection mechanisms for identifying pathological cases within IP-PMM. △ Less

Submitted 8 September, 2021; v1 submitted 27 October, 2020; originally announced October 2020.

Report number: ERGO Technical Report 20-006

A New Preconditioning Approach for an Interior Point-Proximal Method of Multipliers for Linear and Convex Quadratic Programming

Authors: Luca Bergamaschi, Jacek Gondzio, Ángeles Martínez, John W. Pearson, Spyridon Pougkakiotis

Abstract: In this paper, we address the efficient numerical solution of linear and quadratic programming problems, often of large scale. With this aim, we devise an infeasible interior point method, blended with the proximal method of multipliers, which in turn results in a primal-dual regularized interior point method. Application of this method gives rise to a sequence of increasingly ill-conditioned line… ▽ More In this paper, we address the efficient numerical solution of linear and quadratic programming problems, often of large scale. With this aim, we devise an infeasible interior point method, blended with the proximal method of multipliers, which in turn results in a primal-dual regularized interior point method. Application of this method gives rise to a sequence of increasingly ill-conditioned linear systems which cannot always be solved by factorization methods, due to memory and CPU time restrictions. We propose a novel preconditioning strategy which is based on a suitable sparsification of the normal equations matrix in the linear case, and also constitutes the foundation of a block-diagonal preconditioner to accelerate MINRES for linear systems arising from the solution of general quadratic programming problems. Numerical results for a range of test problems demonstrate the robustness of the proposed preconditioning strategy, together with its ability to solve linear systems of very large dimension. △ Less

Submitted 11 December, 2020; v1 submitted 20 December, 2019; originally announced December 2019.

A relaxed interior point method for low-rank semidefinite programming problems with applications to matrix completion

Authors: Stefania Bellavia, Jacek Gondzio, Margherita Porcelli

Abstract: A new relaxed variant of interior point method for low-rank semidefinite programming problems is proposed in this paper. The method is a step outside of the usual interior point framework. In anticipation to converging to a low-rank primal solution, a special nearly low-rank form of all primal iterates is imposed. To accommodate such a (restrictive) structure, the first order optimality conditions… ▽ More A new relaxed variant of interior point method for low-rank semidefinite programming problems is proposed in this paper. The method is a step outside of the usual interior point framework. In anticipation to converging to a low-rank primal solution, a special nearly low-rank form of all primal iterates is imposed. To accommodate such a (restrictive) structure, the first order optimality conditions have to be relaxed and are therefore approximated by solving an auxiliary least-squares problem. The relaxed interior point framework opens numerous possibilities how primal and dual approximated Newton directions can be computed. In particular, it admits the application of both the first- and the second-order methods in this context. The convergence of the method is established. A prototype implementation is discussed and encouraging preliminary computational results are reported for solving the SDP-reformulation of matrix-completion problems. △ Less

Submitted 25 March, 2021; v1 submitted 13 September, 2019; originally announced September 2019.

MSC Class: 90C22; 90C51; 65F10; 65F50

Fast Solution Methods for Convex Quadratic Optimization of Fractional Differential Equations

Authors: Spyridon Pougkakiotis, John W. Pearson, Santolo Leveque, Jacek Gondzio

Abstract: In this paper, we present numerical methods suitable for solving convex quadratic Fractional Differential Equation (FDE) constrained optimization problems, with box constraints on the state and/or control variables. We develop an Alternating Direction Method of Multipliers (ADMM) framework, which uses preconditioned Krylov subspace solvers for the resulting sub-problems. The latter allows us to ta… ▽ More In this paper, we present numerical methods suitable for solving convex quadratic Fractional Differential Equation (FDE) constrained optimization problems, with box constraints on the state and/or control variables. We develop an Alternating Direction Method of Multipliers (ADMM) framework, which uses preconditioned Krylov subspace solvers for the resulting sub-problems. The latter allows us to tackle a range of Partial Differential Equation (PDE) optimization problems with box constraints, posed on space-time domains, that were previously out of the reach of state-of-the-art preconditioners. In particular, by making use of the powerful Generalized Locally Toeplitz (GLT) sequences theory, we show that any existing GLT structure present in the problem matrices is preserved by ADMM, and we propose some preconditioning methodologies that could be used within the solver, to demonstrate the generality of the approach. Focusing on convex quadratic programs with time-dependent 2-dimensional FDE constraints, we derive multilevel circulant preconditioners, which may be embedded within Krylov subspace methods, for solving the ADMM sub-problems. Discretized versions of FDEs involve large dense linear systems. In order to overcome this difficulty, we design a recursive linear algebra, which is based on the Fast Fourier Transform (FFT). We manage to keep the storage requirements linear, with respect to the grid size $N$, while ensuring an order $N \log N$ computational complexity per iteration of the Krylov solver. We implement the proposed method, and demonstrate its scalability, generality, and efficiency, through a series of experiments over different setups of the FDE optimization problem. △ Less

Submitted 6 October, 2020; v1 submitted 31 July, 2019; originally announced July 2019.

Report number: ERGO 19-014

Optimising portfolio diversification and dimensionality

Authors: Mathias Barkhagen, Brian Fleming, Sergio Garcia Quiles, Jacek Gondzio, Joerg Kalcsics, Jens Kroeske, Sotirios Sabanis, Arne Staal

Abstract: A new framework for portfolio diversification is introduced which goes beyond the classical mean-variance approach and portfolio allocation strategies such as risk parity. It is based on a novel concept called portfolio dimensionality that connects diversification to the non-Gaussianity of portfolio returns and can typically be defined in terms of the ratio of risk measures which are homogenous fu… ▽ More A new framework for portfolio diversification is introduced which goes beyond the classical mean-variance approach and portfolio allocation strategies such as risk parity. It is based on a novel concept called portfolio dimensionality that connects diversification to the non-Gaussianity of portfolio returns and can typically be defined in terms of the ratio of risk measures which are homogenous functions of equal degree. The latter arises naturally due to our requirement that diversification measures should be leverage invariant. We introduce this new framework and argue the benefits relative to existing measures of diversification in the literature, before addressing the question of optimizing diversification or, equivalently, dimensionality. Maximising portfolio dimensionality leads to highly non-trivial optimization problems with objective functions which are typically non-convex and potentially have multiple local optima. Two complementary global optimization algorithms are thus presented. For problems of moderate size and more akin to asset allocation problems, a deterministic Branch and Bound algorithm is developed, whereas for problems of larger size a stochastic global optimization algorithm based on Gradient Langevin Dynamics is given. We demonstrate analytically and through numerical experiments that the framework reflects the desired properties often discussed in the literature. △ Less

Submitted 20 September, 2019; v1 submitted 3 June, 2019; originally announced June 2019.

An Interior Point-Proximal Method of Multipliers for Convex Quadratic Programming

Authors: Spyridon Pougkakiotis, Jacek Gondzio

Abstract: In this paper we combine an infeasible Interior Point Method (IPM) with the Proximal Method of Multipliers (PMM). The resulting algorithm (IP-PMM) is interpreted as a primal-dual regularized IPM, suitable for solving linearly constrained convex quadratic programming problems. We apply few iterations of the interior point method to each sub-problem of the proximal method of multipliers. Once a sati… ▽ More In this paper we combine an infeasible Interior Point Method (IPM) with the Proximal Method of Multipliers (PMM). The resulting algorithm (IP-PMM) is interpreted as a primal-dual regularized IPM, suitable for solving linearly constrained convex quadratic programming problems. We apply few iterations of the interior point method to each sub-problem of the proximal method of multipliers. Once a satisfactory solution of the PMM sub-problem is found, we update the PMM parameters, form a new IPM neighbourhood and repeat this process. Given this framework, we prove polynomial complexity of the algorithm, under standard assumptions. To our knowledge, this is the first polynomial complexity result for a primal-dual regularized IPM. The algorithm is guided by the use of a single penalty parameter; that of the logarithmic barrier. In other words, we show that IP-PMM inherits the polynomial complexity of IPMs, as well as the strict convexity of the PMM sub-problems. The updates of the penalty parameter are controlled by IPM, and hence are well-tuned, and do not depend on the problem solved. Furthermore, we study the behavior of the method when it is applied to an infeasible problem, and identify a necessary condition for infeasibility. The latter is used to construct an infeasibility detection mechanism. Subsequently, we provide a robust implementation of the presented algorithm and test it over a set of small to large scale linear and convex quadratic programming problems. The numerical results demonstrate the benefits of using regularization in IPMs as well as the reliability of the method. △ Less

Submitted 5 March, 2020; v1 submitted 23 April, 2019; originally announced April 2019.

Dynamic Non-Diagonal Regularization in Interior Point Methods for Linear and Convex Quadratic Programming

Authors: Spyridon Pougkakiotis, Jacek Gondzio

Abstract: In this paper, we present a dynamic non-diagonal regularization for interior point methods. The non-diagonal aspect of this regularization is implicit, since all the off-diagonal elements of the regularization matrices are cancelled out by those elements present in the Newton system, which do not contribute important information in the computation of the Newton direction. Such a regularization has… ▽ More In this paper, we present a dynamic non-diagonal regularization for interior point methods. The non-diagonal aspect of this regularization is implicit, since all the off-diagonal elements of the regularization matrices are cancelled out by those elements present in the Newton system, which do not contribute important information in the computation of the Newton direction. Such a regularization has multiple goals. The obvious one is to improve the spectral properties of the Newton system solved at each iteration of the interior point method. On the other hand, the regularization matrices introduce sparsity to the aforementioned linear system, allowing for more efficient factorizations. We also propose a rule for tuning the regularization dynamically based on the properties of the problem, such that sufficiently large eigenvalues of the non-regularized system are perturbed insignificantly. This alleviates the need of finding specific regularization values through experimentation, which is the most common approach in literature. We provide perturbation bounds for the eigenvalues of the non-regularized system matrix and then discuss the spectral properties of the regularized matrix. Finally, we demonstrate the efficiency of the method applied to solve standard small and medium-scale linear and convex quadratic programming test problems. △ Less

Submitted 13 February, 2019; originally announced February 2019.

Mathematical models for stable matching problems with ties and incomplete lists

Authors: Maxence Delorme, Sergio García, Jacek Gondzio, Joerg Kalcsics, David Manlove, William Pettersson

Abstract: We present new integer linear programming (ILP) models for NP-hard optimisation problems in instances of the Stable Marriage problem with Ties and Incomplete lists (SMTI) and its many-to-one generalisation, the Hospitals / Residents problem with Ties (HRT). These models can be used to efficiently solve these optimisation problems when applied to (i) instances derived from real-world applications,… ▽ More We present new integer linear programming (ILP) models for NP-hard optimisation problems in instances of the Stable Marriage problem with Ties and Incomplete lists (SMTI) and its many-to-one generalisation, the Hospitals / Residents problem with Ties (HRT). These models can be used to efficiently solve these optimisation problems when applied to (i) instances derived from real-world applications, and (ii) larger instances that are randomly-generated. In the case of SMTI, we consider instances arising from the pairing of children with adoptive families, where preferences are obtained from a quality measure of each possible pairing of child to family. In this case we seek a maximum weight stable matching. We present new algorithms for preprocessing instances of SMTI with ties on both sides, as well as new ILP models. Algorithms based on existing state-of-the-art models only solve 6 of our 22 real-world instances within an hour per instance, and our new models solve all 22 instances within a mean runtime of 60 seconds. For HRT, we consider instances derived from the problem of assigning junior doctors to foundation posts in Scottish hospitals. Here we seek a maximum size stable matching. We show how to extend our models for SMTI to the HRT case. For the real instances, we reduce the mean runtime from an average of 144 seconds when using state-of-the-art methods, to 3 seconds when using our new ILP-based algorithms. We also show that our models outperform considerably state-of-the-art models on larger randomly-generated instances of SMTI and HRT. △ Less

Submitted 17 March, 2019; v1 submitted 5 October, 2018; originally announced October 2018.

Comments: 31 pages, 11 tables, 1 figure

Report number: ERGO-18-024 MSC Class: 90C27

Global Solutions of Nonconvex Standard Quadratic Programs via Mixed Integer Linear Programming Reformulations

Authors: Jacek Gondzio, E. Alper Yildirim

Abstract: A standard quadratic program is an optimization problem that consists of minimizing a (nonconvex) quadratic form over the unit simplex. We focus on reformulating a standard quadratic program as a mixed integer linear programming problem. We propose two alternative mixed integer linear programming formulations. Our first formulation is based on casting a standard quadratic program as a linear progr… ▽ More A standard quadratic program is an optimization problem that consists of minimizing a (nonconvex) quadratic form over the unit simplex. We focus on reformulating a standard quadratic program as a mixed integer linear programming problem. We propose two alternative mixed integer linear programming formulations. Our first formulation is based on casting a standard quadratic program as a linear program with complementarity constraints. We then employ binary variables to linearize the complementarity constraints. For the second formulation, we first derive an overestimating function of the objective function and establish its tightness at any global minimizer. We then linearize the overestimating function using binary variables and obtain our second formulation. For both formulations, we propose a set of valid inequalities. Our extensive computational results illustrate that the proposed mixed integer linear programming reformulations significantly outperform other global solution approaches. On larger instances, we usually observe improvements of orders of magnitude. △ Less

Submitted 4 October, 2018; originally announced October 2018.

Comments: 40 pages, 3 figures

Report number: Technical Report, ERGO-18-022, University of Edinburgh, Edinburgh, United Kingdom MSC Class: 90C20; 90C11; 90C26

Quasi-Newton approaches to Interior Point Methods for quadratic problems

Authors: J. Gondzio, F. N. C. Sobral

Abstract: Interior Point Methods (IPM) rely on the Newton method for solving systems of nonlinear equations. Solving the linear systems which arise from this approach is the most computationally expensive task of an interior point iteration. If, due to problem's inner structure, there are special techniques for efficiently solving linear systems, IPMs enjoy fast convergence and are able to solve large scale… ▽ More Interior Point Methods (IPM) rely on the Newton method for solving systems of nonlinear equations. Solving the linear systems which arise from this approach is the most computationally expensive task of an interior point iteration. If, due to problem's inner structure, there are special techniques for efficiently solving linear systems, IPMs enjoy fast convergence and are able to solve large scale optimization problems. It is tempting to try to replace the Newton method by quasi-Newton methods. Quasi-Newton approaches to IPMs either are built to approximate the Lagrangian function for nonlinear programming problems or provide an inexpensive preconditioner. In this work we study the impact of using quasi-Newton methods applied directly to the nonlinear system of equations for general quadratic programming problems. The cost of each iteration can be compared to the cost of computing correctors in a usual interior point iteration. Numerical experiments show that the new approach is able to reduce the overall number of matrix factorizations and is suitable for a matrix-free implementation. △ Less

Submitted 26 June, 2018; originally announced June 2018.

Report number: ERGO 18-015 MSC Class: 90C30; 90C51; 90C53

Rank Revealing Gaussian Elimination by the Maximum Volume Concept

Authors: Lukas Schork, Jacek Gondzio

Abstract: A Gaussian elimination algorithm is presented that reveals the numerical rank of a matrix by yielding small entries in the Schur complement. The algorithm uses the maximum volume concept to find a square nonsingular submatrix of maximum dimension. The bounds on the revealed singular values are similar to the best known bounds for rank revealing LU factorization, but in contrast to existing methods… ▽ More A Gaussian elimination algorithm is presented that reveals the numerical rank of a matrix by yielding small entries in the Schur complement. The algorithm uses the maximum volume concept to find a square nonsingular submatrix of maximum dimension. The bounds on the revealed singular values are similar to the best known bounds for rank revealing LU factorization, but in contrast to existing methods the algorithm does not make use of the normal matrix. An implementation for dense matrices is described whose computational cost is roughly twice the cost of an LU factorization with complete pivoting. Because of its flexibility in choosing pivot elements, the algorithm is amenable to implementation with blocked memory access and for sparse matrices. △ Less

Submitted 8 February, 2018; originally announced February 2018.

Report number: ERGO-18-002

An inexact potential reduction method for linear programming

Authors: Lukas Schork, Jacek Gondzio

Abstract: A class of interior point methods using inexact directions is analysed. The linear system arising in interior point methods for linear programming is reformulated such that the solution is less sensitive to perturbations in the right-hand side. For the new system an implementable condition is formulated that controls the relative error in the solution. Based on this condition, a feasible and an in… ▽ More A class of interior point methods using inexact directions is analysed. The linear system arising in interior point methods for linear programming is reformulated such that the solution is less sensitive to perturbations in the right-hand side. For the new system an implementable condition is formulated that controls the relative error in the solution. Based on this condition, a feasible and an infeasible potential reduction method are described which retain the convergence and complexity bounds known for exact directions. △ Less

Submitted 29 July, 2016; originally announced August 2016.

Report number: ERGO-16-005 MSC Class: 90C05

Performance of First- and Second-Order Methods for L1-Regularized Least Squares Problems

Authors: Kimon Fountoulakis, Jacek Gondzio

Abstract: We study the performance of first- and second-order optimization methods for l1-regularized sparse least-squares problems as the conditioning of the problem changes and the dimensions of the problem increase up to one trillion. A rigorously defined generator is presented which allows control of the dimensions, the conditioning and the sparsity of the problem. The generator has very low memory requ… ▽ More We study the performance of first- and second-order optimization methods for l1-regularized sparse least-squares problems as the conditioning of the problem changes and the dimensions of the problem increase up to one trillion. A rigorously defined generator is presented which allows control of the dimensions, the conditioning and the sparsity of the problem. The generator has very low memory requirements and scales well with the dimensions of the problem. △ Less

Submitted 14 December, 2015; v1 submitted 11 March, 2015; originally announced March 2015.

Comments: 32 pages, 11 figures

Report number: Technical Report ERGO 15-005

A Preconditioner for a Primal-Dual Newton Conjugate Gradients Method for Compressed Sensing Problems

Authors: Ioannis Dassios, Kimon Fountoulakis, Jacek Gondzio

Abstract: In this paper we are concerned with the solution of Compressed Sensing (CS) problems where the signals to be recovered are sparse in coherent and redundant dictionaries. We extend a primal-dual Newton Conjugate Gradients (pdNCG) method for CS problems. We provide an inexpensive and provably effective preconditioning technique for linear systems using pdNCG. Numerical results are presented on CS pr… ▽ More In this paper we are concerned with the solution of Compressed Sensing (CS) problems where the signals to be recovered are sparse in coherent and redundant dictionaries. We extend a primal-dual Newton Conjugate Gradients (pdNCG) method for CS problems. We provide an inexpensive and provably effective preconditioning technique for linear systems using pdNCG. Numerical results are presented on CS problems which demonstrate the performance of pdNCG with the proposed preconditioner compared to state-of-the-art existing solvers. △ Less

Submitted 2 July, 2015; v1 submitted 30 December, 2014; originally announced January 2015.

Comments: 31 pages, 36 figures, 7 tables. arXiv admin note: substantial text overlap with arXiv:1405.4146

Report number: Technical Report ERGO 14-021

Action constrained quasi-Newton methods

Authors: Robert Mansel Gower, Jacek Gondzio

Abstract: At the heart of Newton based optimization methods is a sequence of symmetric linear systems. Each consecutive system in this sequence is similar to the next, so solving them separately is a waste of computational effort. Here we describe automatic preconditioning techniques for iterative methods for solving such sequences of systems by maintaining an estimate of the inverse system matrix. We updat… ▽ More At the heart of Newton based optimization methods is a sequence of symmetric linear systems. Each consecutive system in this sequence is similar to the next, so solving them separately is a waste of computational effort. Here we describe automatic preconditioning techniques for iterative methods for solving such sequences of systems by maintaining an estimate of the inverse system matrix. We update the estimate of the inverse system matrix with quasi-Newton type formulas based on what we call an action constraint instead of the secant equation. We implement the estimated inverses as preconditioners in a Newton-CG method and prove quadratic termination. Our implementation is the first parallel quasi-Newton preconditioners, in full and limited memory variants. Tests on logistic Support Vector Machine problems reveal that our method is very efficient, converging in wall clock time before a Newton-CG method without preconditioning. Further tests on a set of classic test problems reveal that the method is robust. The action constraint makes these updates flexible enough to mesh with trust-region and active set methods, a flexibility that is not present in classic quasi-Newton methods. △ Less

Submitted 27 December, 2014; originally announced December 2014.

MSC Class: 65K05; 49M37; 90C25; 90C53 ACM Class: G.1.6

A Second-Order Method for Compressed Sensing Problems with Coherent and Redundant Dictionaries

Authors: Ioannis Dassios, Kimon Fountoulakis, Jacek Gondzio

Abstract: In this paper we are interested in the solution of Compressed Sensing (CS) problems where the signals to be recovered are sparse in coherent and redundant dictionaries. CS problems of this type are convex with non-smooth and non-separable regularization term, therefore a specialized solver is required. We propose a primal-dual Newton Conjugate Gradients (pdNCG) method. We prove global convergence… ▽ More In this paper we are interested in the solution of Compressed Sensing (CS) problems where the signals to be recovered are sparse in coherent and redundant dictionaries. CS problems of this type are convex with non-smooth and non-separable regularization term, therefore a specialized solver is required. We propose a primal-dual Newton Conjugate Gradients (pdNCG) method. We prove global convergence and fast local rate of convergence for pdNCG. Moreover, well-known properties of CS problems are exploited for the development of provably effective preconditioning techniques that speed-up the approximate solution of linear systems which arise. Numerical results are presented on CS problems which demonstrate the performance of pdNCG compared to a state-of-the-art existing solver. △ Less

Submitted 30 May, 2014; v1 submitted 16 May, 2014; originally announced May 2014.

Comments: 26 pages, 22 figures

Report number: Technical Report ERGO-14-007

Large-scale optimization with the primal-dual column generation method

Authors: Jacek Gondzio, Pablo González-Brevis, Pedro Munari

Abstract: The primal-dual column generation method (PDCGM) is a general-purpose column generation technique that relies on the primal-dual interior point method to solve the restricted master problems. The use of this interior point method variant allows to obtain suboptimal and well-centered dual solutions which naturally stabilizes the column generation. As recently presented in the literature, reductions… ▽ More The primal-dual column generation method (PDCGM) is a general-purpose column generation technique that relies on the primal-dual interior point method to solve the restricted master problems. The use of this interior point method variant allows to obtain suboptimal and well-centered dual solutions which naturally stabilizes the column generation. As recently presented in the literature, reductions in the number of calls to the oracle and in the CPU times are typically observed when compared to the standard column generation, which relies on extreme optimal dual solutions. However, these results are based on relatively small problems obtained from linear relaxations of combinatorial applications. In this paper, we investigate the behaviour of the PDCGM in a broader context, namely when solving large-scale convex optimization problems. We have selected applications that arise in important real-life contexts such as data analysis (multiple kernel learning problem), decision-making under uncertainty (two-stage stochastic programming problems) and telecommunication and transportation networks (multicommodity network flow problem). In the numerical experiments, we use publicly available benchmark instances to compare the performance of the PDCGM against recent results for different methods presented in the literature, which were the best available results to date. The analysis of these results suggests that the PDCGM offers an attractive alternative over specialized methods since it remains competitive in terms of number of iterations and CPU times even for large-scale optimization problems. △ Less

Submitted 16 February, 2015; v1 submitted 9 September, 2013; originally announced September 2013.

Comments: 28 pages, 1 figure, minor revision, scaled CPU times

Report number: Technical Report ERGO 13-014 MSC Class: 90C25; 90C06; 90C51; 90C90; 90C05; 49M27; 65K05

A Second-Order Method for Strongly Convex L1-Regularization Problems

Authors: Kimon Fountoulakis, Jacek Gondzio

Abstract: In this paper a robust second-order method is developed for the solution of strongly convex l1-regularized problems. The main aim is to make the proposed method as inexpensive as possible, while even difficult problems can be efficiently solved. The proposed approach is a primal-dual Newton Conjugate Gradients (pdNCG) method. Convergence properties of pdNCG are studied and worst-case iteration com… ▽ More In this paper a robust second-order method is developed for the solution of strongly convex l1-regularized problems. The main aim is to make the proposed method as inexpensive as possible, while even difficult problems can be efficiently solved. The proposed approach is a primal-dual Newton Conjugate Gradients (pdNCG) method. Convergence properties of pdNCG are studied and worst-case iteration complexity is established. Numerical results are presented on synthetic sparse least-squares problems and real world machine learning problems. △ Less

Submitted 11 January, 2015; v1 submitted 23 June, 2013; originally announced June 2013.

Comments: 30 pages, 13 figures, 1 table

Report number: Technical Report ERGO-14-005 MSC Class: 68W40; 65K05; 90C06; 90C25; 90C30; 90C51

Inexact Coordinate Descent: Complexity and Preconditioning

Authors: Rachael Tappenden, Peter Richtárik, Jacek Gondzio

Abstract: In this paper we consider the problem of minimizing a convex function using a randomized block coordinate descent method. One of the key steps at each iteration of the algorithm is determining the update to a block of variables. Existing algorithms assume that in order to compute the update, a particular subproblem is solved exactly. In his work we relax this requirement, and allow for the subprob… ▽ More In this paper we consider the problem of minimizing a convex function using a randomized block coordinate descent method. One of the key steps at each iteration of the algorithm is determining the update to a block of variables. Existing algorithms assume that in order to compute the update, a particular subproblem is solved exactly. In his work we relax this requirement, and allow for the subproblem to be solved inexactly, leading to an inexact block coordinate descent method. Our approach incorporates the best known results for exact updates as a special case. Moreover, these theoretical guarantees are complemented by practical considerations: the use of iterative techniques to determine the update as well as the use of preconditioning for further acceleration. △ Less

Submitted 10 December, 2014; v1 submitted 19 April, 2013; originally announced April 2013.

Comments: 32 pages, 6 tables, 2 figures, 1 algorithm

Convergence Analysis of an Inexact Feasible Interior Point Method for Convex Quadratic Programming

Authors: Jacek Gondzio

Abstract: In this paper we will discuss two variants of an inexact feasible interior point algorithm for convex quadratic programming. We will consider two different neighbourhoods: a (small) one induced by the use of the Euclidean norm which yields a short-step algorithm and a symmetric one induced by the use of the infinity norm which yields a (practical) long-step algorithm. Both algorithms allow for the… ▽ More In this paper we will discuss two variants of an inexact feasible interior point algorithm for convex quadratic programming. We will consider two different neighbourhoods: a (small) one induced by the use of the Euclidean norm which yields a short-step algorithm and a symmetric one induced by the use of the infinity norm which yields a (practical) long-step algorithm. Both algorithms allow for the Newton equation system to be solved inexactly. For both algorithms we will provide conditions for the level of error acceptable in the Newton equation and establish the worst-case complexity results. △ Less

Submitted 29 August, 2012; originally announced August 2012.

Report number: Technical Report ERGO-2012-008

Matrix-free Interior Point Method for Compressed Sensing Problems

Authors: Kimon Fountoulakis, Jacek Gondzio, Pavel Zhlobich

Abstract: We consider a class of optimization problems for sparse signal reconstruction which arise in the field of Compressed Sensing (CS). A plethora of approaches and solvers exist for such problems, for example GPSR, FPC AS, SPGL1, NestA, $\ell_{1}_\ell_{s}$, PDCO to mention a few. Compressed Sensing applications lead to very well conditioned optimization problems and therefore can be solved easily by s… ▽ More We consider a class of optimization problems for sparse signal reconstruction which arise in the field of Compressed Sensing (CS). A plethora of approaches and solvers exist for such problems, for example GPSR, FPC AS, SPGL1, NestA, $\ell_{1}_\ell_{s}$, PDCO to mention a few. Compressed Sensing applications lead to very well conditioned optimization problems and therefore can be solved easily by simple first-order methods. Interior point methods (IPMs) rely on the Newton method hence they use the second-order information. They have numerous advantageous features and one clear drawback: being the second-order approach they need to solve linear equations and this operation has (in the general dense case) an $O(n^3)$ computational complexity. Attempts have been made to specialize IPMs to sparse reconstruction problems and they have led to interesting developments implemented in $\ell_1\_\ell_s$ and PDCO softwares. We go a few steps further. First, we use the matrix-free interior point method, an approach which redesigns IPM to avoid the need to explicitly formulate (and store) the Newton equation systems. Secondly, we exploit the special features of the signal processing matrices within the matrix-free IPM. Two such features are of particular interest: an excellent conditioning of these matrices and the ability to perform inexpensive (low complexity) matrix-vector multiplications with them. Computational experience with large scale one-dimensional signals confirms that the new approach is efficient and offers an attractive alternative to other state-of-the-art solvers. △ Less

Submitted 16 November, 2013; v1 submitted 27 August, 2012; originally announced August 2012.

Report number: Technical Report ERGO 12-006 MSC Class: 90C05; 90C06; 90C30; 90C25; 90C51

Solving Large-Scale Optimization Problems Related to Bell's Theorem

Authors: Jacek Gondzio, Jacek A. Gruca, J. A. Julian Hall, Wiesław Laskowski, Marek Żukowski

Abstract: Impossibility of finding local realistic models for quantum correlations due to entanglement is an important fact in foundations of quantum physics, gaining now new applications in quantum information theory. We present an in-depth description of a method of testing the existence of such models, which involves two levels of optimization: a higher-level non-linear task and a lower-level linear prog… ▽ More Impossibility of finding local realistic models for quantum correlations due to entanglement is an important fact in foundations of quantum physics, gaining now new applications in quantum information theory. We present an in-depth description of a method of testing the existence of such models, which involves two levels of optimization: a higher-level non-linear task and a lower-level linear programming (LP) task. The article compares the performances of the existing implementation of the method, where the LPs are solved with the simplex method, and our new implementation, where the LPs are solved with a matrix-free interior point method. We describe in detail how the latter can be applied to our problem, discuss the basic scenario and possible improvements and how they impact on overall performance. Significant performance advantage of the matrix-free interior point method over the simplex method is confirmed by extensive computational results. The new method is able to solve problems which are orders of magnitude larger. Consequently, the noise resistance of the non-classicality of correlations of several types of quantum states, which has never been computed before, can now be efficiently determined. An extensive set of data in the form of tables and graphics is presented and discussed. The article is intended for all audiences, no quantum-mechanical background is necessary. △ Less

Submitted 18 January, 2014; v1 submitted 16 April, 2012; originally announced April 2012.

Comments: 19 pages, 7 tables, 1 figure

Report number: Technical Report ERGO-2012-004 MSC Class: 90C51 (Primary) 90C05; 90C06; 90C90 (Secondary)

Journal ref: Journal of Computational and Applied Mathematics 263, 392 (2014)

Multilevel quasiseparable matrices in PDE-constrained optimization

Authors: Jacek Gondzio, Pavel Zhlobich

Abstract: Optimization problems with constraints in the form of a partial differential equation arise frequently in the process of engineering design. The discretization of PDE-constrained optimization problems results in large-scale linear systems of saddle-point type. In this paper we propose and develop a novel approach to solving such systems by exploiting so-called quasiseparable matrices. One may thin… ▽ More Optimization problems with constraints in the form of a partial differential equation arise frequently in the process of engineering design. The discretization of PDE-constrained optimization problems results in large-scale linear systems of saddle-point type. In this paper we propose and develop a novel approach to solving such systems by exploiting so-called quasiseparable matrices. One may think of a usual quasiseparable matrix as of a discrete analog of the Green's function of a one-dimensional differential operator. Nice feature of such matrices is that almost every algorithm which employs them has linear complexity. We extend the application of quasiseparable matrices to problems in higher dimensions. Namely, we construct a class of preconditioners which can be computed and applied at a linear computational cost. Their use with appropriate Krylov methods leads to algorithms of nearly linear complexity. △ Less

Submitted 30 December, 2011; v1 submitted 27 December, 2011; originally announced December 2011.

MSC Class: 49M25; 49K20; 65F08; 65F10; 65F50; 65N22