Showing 1–37 of 37 results for author: Tam, M K

A general approach to distributed operator splitting

Authors: Minh N. Dao, Matthew K. Tam, Thang D. Truong

Abstract: Splitting methods have emerged as powerful tools to address complex problems by decomposing them into smaller solvable components. In this work, we develop a general approach of forward-backward splitting methods for solving monotone inclusion problems involving both set-valued and single-valued operators, where the latter may lack cocoercivity. Our proposed approach, based on some coefficient mat… ▽ More Splitting methods have emerged as powerful tools to address complex problems by decomposing them into smaller solvable components. In this work, we develop a general approach of forward-backward splitting methods for solving monotone inclusion problems involving both set-valued and single-valued operators, where the latter may lack cocoercivity. Our proposed approach, based on some coefficient matrices, not only encompasses several important existing algorithms but also extends to new ones, offering greater flexibility for different applications. Moreover, by appropriately selecting the coefficient matrices, the resulting algorithms can be implemented in a distributed and decentralized manner. △ Less

Submitted 21 April, 2025; originally announced April 2025.

The Golden Ratio Primal-Dual Algorithm with Two New Stepsize Rules for Convex-Concave Saddle Point Problems

Authors: Santanu Soe, Matthew K. Tam, V. Vetrivel

Abstract: In this paper, we present two stepsize rules for the extended Golden Ratio primal-dual algorithm (E-GRPDA) designed to address structured convex optimization problems in finite-dimensional real Hilbert spaces. The first rule features a nonincreasing primal stepsize that remains bounded below by a positive constant and is updated adaptively at each iteration, eliminating the need for the Lipschitz… ▽ More In this paper, we present two stepsize rules for the extended Golden Ratio primal-dual algorithm (E-GRPDA) designed to address structured convex optimization problems in finite-dimensional real Hilbert spaces. The first rule features a nonincreasing primal stepsize that remains bounded below by a positive constant and is updated adaptively at each iteration, eliminating the need for the Lipschitz constant of the gradient of the function and the norm of the operator involved. The second stepsize rule is adaptive, adjusting based on the local smoothness of the smooth component function and the local estimate of the norm of the operator. In other words, we present an adaptive version of the E-GRPDA algorithm. Importantly, both methods avoid the use of backtracking to estimate the operator norm. We prove that E-GRPDA achieves an ergodic sublinear convergence rate with both stepsize rules, evaluated using the primal-dual gap function. Additionally, we establish an R-linear convergence rate for E-GRPDA with the first stepsize rule, under some standard assumptions and with appropriately chosen parameters. Through numerical experiments on various convex optimization problems, we demonstrate the effectiveness of our approaches and compare their performance to the existing ones. △ Less

Submitted 25 February, 2025; originally announced February 2025.

Comments: 36 pages, 32 figures

MSC Class: 65K10; 65Y20; 49M29; 49K35; 49J52

Linear Convergence of Resolvent Splitting with Minimal Lifting and its Application to a Primal-Dual Algorithm

Authors: Farhana A. Simi, Matthew K. Tam

Abstract: We consider resolvent splitting algorithms for finding a zero of the sum of finitely many maximally monotone operators. The standard approach to solving this type of problem involves reformulating as a two-operator problem in the product-space and applying the Douglas-Rachford algorithm. However, existing results for linear convergence cannot be applied in the product-space formulation due to a la… ▽ More We consider resolvent splitting algorithms for finding a zero of the sum of finitely many maximally monotone operators. The standard approach to solving this type of problem involves reformulating as a two-operator problem in the product-space and applying the Douglas-Rachford algorithm. However, existing results for linear convergence cannot be applied in the product-space formulation due to a lack of appropriate Lipschitz continuity and strong monotonicity. In this work, we investigate a different approach that does not rely on the Douglas-Rachford algorithm or the product-space directly. We establish linear convergence of the "resolvent splitting with minimal lifting" algorithm due to Malitsky and Tam for monotone inclusions with finitely many operators. Our results are then used to derive linear convergence of a primal-dual algorithm for convex minimization problems involving infimal convolutions. The theoretical results are demonstrated on numerical experiments in image denoising. △ Less

Submitted 17 December, 2024; originally announced December 2024.

MSC Class: 47H05; 49M27; 65K10; 90C30

A distributed proximal splitting method with linesearch for locally Lipschitz gradients

Authors: Felipe Atenas, Minh N. Dao, Matthew K. Tam

Abstract: In this paper, we propose a distributed first-order algorithm with backtracking linesearch for solving multi-agent minimisation problems, where each agent handles a local objective involving nonsmooth and smooth components. Unlike existing methods that require global Lipschitz continuity and predefined stepsizes, our algorithm adjusts stepsizes using distributed linesearch procedures, making it su… ▽ More In this paper, we propose a distributed first-order algorithm with backtracking linesearch for solving multi-agent minimisation problems, where each agent handles a local objective involving nonsmooth and smooth components. Unlike existing methods that require global Lipschitz continuity and predefined stepsizes, our algorithm adjusts stepsizes using distributed linesearch procedures, making it suitable for problems where global constants are unavailable or difficult to compute. The proposed algorithm is designed within an abstract linesearch framework for a primal-dual proximal-gradient method to solve min-max convex-concave problems, enabling the consensus constraint to be decoupled from the optimisation task. Our theoretical analysis allows for gradients of functions to be locally Lipschitz continuous, relaxing the prevalent assumption of globally Lipschitz continuous gradients. △ Less

Submitted 13 May, 2025; v1 submitted 20 October, 2024; originally announced October 2024.

Comments: 31 pages

MSC Class: 90C25; 68W15; 49M27; 65K05

A First-Order Algorithm for Decentralised Min-Max Problems

Authors: Yura Malitsky, Matthew K. Tam

Abstract: In this work, we consider a connected network of finitely many agents working cooperatively to solve a min-max problem with convex-concave structure. We propose a decentralised first-order algorithm which can be viewed as a non-trivial combination of two algorithms: PG-EXTRA for decentralised minimisation problems and the forward reflected backward method for (non-distributed) min-max problems. In… ▽ More In this work, we consider a connected network of finitely many agents working cooperatively to solve a min-max problem with convex-concave structure. We propose a decentralised first-order algorithm which can be viewed as a non-trivial combination of two algorithms: PG-EXTRA for decentralised minimisation problems and the forward reflected backward method for (non-distributed) min-max problems. In each iteration of our algorithm, each agent computes the gradient of the smooth component of its local objective function as well as the proximal operator of its nonsmooth component, following by a round of communication with its neighbours. Our analysis shows that the sequence generated by the method converges under standard assumptions with non-decaying stepsize. △ Less

Submitted 22 August, 2023; originally announced August 2023.

Comments: 15 pages, 1 figure

Frugal and Decentralised Resolvent Splittings Defined by Nonexpansive Operators

Authors: Matthew K. Tam

Abstract: Frugal resolvent splittings are a class of fixed point algorithms for finding a zero in the sum of the sum of finitely many set-valued monotone operators, where the fixed point operator uses only vector addition, scalar multiplication and the resolvent of each monotone operator once per iteration. In the literature, the convergence analyses of these schemes are performed in an inefficient, algorit… ▽ More Frugal resolvent splittings are a class of fixed point algorithms for finding a zero in the sum of the sum of finitely many set-valued monotone operators, where the fixed point operator uses only vector addition, scalar multiplication and the resolvent of each monotone operator once per iteration. In the literature, the convergence analyses of these schemes are performed in an inefficient, algorithm-by-algorithm basis. In this work, we address this by developing a general framework for frugal resolvent splitting which simultaneously covers and extends several important schemes in the literature. The framework also yields a new resolvent splitting algorithm which is suitable for decentralised implementation on regular networks. △ Less

Submitted 4 June, 2023; v1 submitted 8 November, 2022; originally announced November 2022.

Comments: 14 pages

Single-Projection Procedure for Infinite Dimensional Convex Optimization Problems

Authors: Hoa T. Bui, Regina S. Burachik, Evgeni A. Nurminski, Matthew K. Tam

Abstract: In this work, we consider a class of convex optimization problems in a real Hilbert space that can be solved by performing a single projection, i.e., by projecting an infeasible point onto the feasible set. Our results improve those established for the linear programming setting in Nurminski (2015) by considering problems that: (i) may have multiple solutions, (ii) do not satisfy strict complement… ▽ More In this work, we consider a class of convex optimization problems in a real Hilbert space that can be solved by performing a single projection, i.e., by projecting an infeasible point onto the feasible set. Our results improve those established for the linear programming setting in Nurminski (2015) by considering problems that: (i) may have multiple solutions, (ii) do not satisfy strict complementary conditions, and (iii) possess non-linear convex constraints. As a by-product of our analysis, we provide a quantitative estimate on the required distance between the infeasible point and the feasible set in order for its projection to be a solution of the problem. Our analysis relies on a "sharpness" property of the constraint set; a new property we introduce here. △ Less

Submitted 9 April, 2024; v1 submitted 20 October, 2022; originally announced October 2022.

Bregman Golden Ratio Algorithms for Variational Inequalities

Authors: Matthew K. Tam, Daniel J. Uteda

Abstract: Variational inequalities provide a framework through which many optimisation problems can be solved, in particular, saddle-point problems. In this paper, we study modifications to the so-called Golden RAtio ALgorithm (GRAAL) for variational inequalities -- a method which uses a fully explicit adaptive step-size, and provides convergence results under local Lipschitz assumptions without requiring b… ▽ More Variational inequalities provide a framework through which many optimisation problems can be solved, in particular, saddle-point problems. In this paper, we study modifications to the so-called Golden RAtio ALgorithm (GRAAL) for variational inequalities -- a method which uses a fully explicit adaptive step-size, and provides convergence results under local Lipschitz assumptions without requiring backtracking. We present and analyse two Bregman modifications to GRAAL: the first uses a fixed step-size and converges under global Lipschitz assumptions, and the second uses an adaptive step-size rule. Numerical performance of the former method is demonstrated on a bimatrix game arising in network communication, and of the latter on two problems, namely, power allocation in Gaussian communication channels and $N$-person Cournot completion games. In all of these applications, an appropriately chosen Bregman distance simplifies the projection steps computed as part of the algorithm. △ Less

Submitted 9 August, 2022; originally announced August 2022.

Comments: 24 pages

MSC Class: 47J20; 49J40; 65K15; 65Y20

Distributed Forward-Backward Methods for Ring Networks

Authors: Francisco J. Aragón-Artacho, Yura Malitsky, Matthew K. Tam, David Torregrosa-Belén

Abstract: In this work, we propose and analyse forward-backward-type algorithms for finding a zero of the sum of finitely many monotone operators, which are not based on reduction to a two operator inclusion in the product space. Each iteration of the studied algorithms requires one resolvent evaluation per set-valued operator, one forward evaluation per cocoercive operator, and two forward evaluations per… ▽ More In this work, we propose and analyse forward-backward-type algorithms for finding a zero of the sum of finitely many monotone operators, which are not based on reduction to a two operator inclusion in the product space. Each iteration of the studied algorithms requires one resolvent evaluation per set-valued operator, one forward evaluation per cocoercive operator, and two forward evaluations per monotone operator. Unlike existing methods, the structure of the proposed algorithms are suitable for distributed, decentralised implementation in ring networks without needing global summation to enforce consensus between nodes. △ Less

Submitted 12 July, 2022; v1 submitted 30 November, 2021; originally announced December 2021.

Comments: 19 pages

Resolvent Splitting for Sums of Monotone Operators with Minimal Lifting

Authors: Yura Malitsky, Matthew K. Tam

Abstract: In this work, we study fixed point algorithms for finding a zero in the sum of $n\geq 2$ maximally monotone operators by using their resolvents. More precisely, we consider the class of such algorithms where each resolvent is evaluated only once per iteration. For any algorithm from this class, we show that the underlying fixed point operator is necessarily defined on a $d$-fold Cartesian product… ▽ More In this work, we study fixed point algorithms for finding a zero in the sum of $n\geq 2$ maximally monotone operators by using their resolvents. More precisely, we consider the class of such algorithms where each resolvent is evaluated only once per iteration. For any algorithm from this class, we show that the underlying fixed point operator is necessarily defined on a $d$-fold Cartesian product space with $d\geq n-1$. Further, we show that this bound is unimprovable by providing a family of examples for which $d=n-1$ is attained. This family includes the Douglas-Rachford algorithm as the special case when $n=2$. Applications of the new family of algorithms in distributed decentralised optimisation and multi-block extensions of the alternation direction method of multipliers (ADMM) are discussed. △ Less

Submitted 21 July, 2022; v1 submitted 5 August, 2021; originally announced August 2021.

Comments: 23 pages

Minimal spin deflection of Kerr-Newman and Supersymmetric black hole

Authors: Bo-Ting Chen, Ming-Zhi Chung, Yu-tin Huang, Man Kuan Tam

Abstract: Recent studies have shown that minimal couplings for massive spinning particles, which in the classical limit reproduce the leading PM Kerr black hole dynamics, leads to an Eikonal S-matrix exhibiting spin-entanglement suppression. In this paper we trace this phenomenon to the suppression of spin-flipping components in the S-matrix, known to be the hallmark of minimal coupling in the ultra-relativ… ▽ More Recent studies have shown that minimal couplings for massive spinning particles, which in the classical limit reproduce the leading PM Kerr black hole dynamics, leads to an Eikonal S-matrix exhibiting spin-entanglement suppression. In this paper we trace this phenomenon to the suppression of spin-flipping components in the S-matrix, known to be the hallmark of minimal coupling in the ultra-relativistic limit. We further generalize the consideration to charged and $\mathcal{N}=4$ black holes, demonstrating that in both cases maximal suppression occurs at the extremal limit. △ Less

Submitted 2 December, 2021; v1 submitted 23 June, 2021; originally announced June 2021.

Comments: 28 pages, 6 figures, 1 ancillary file

Method of Alternating Projection for the Absolute Value Equation

Authors: Jan Harold Alcantara, Jein-Shan Chen, Matthew K. Tam

Abstract: A novel approach for solving the general absolute value equation $Ax+B|x| = c$ where $A,B\in \mathbb{R}^{m\times n}$ and $c\in \mathbb{R}^m$ is presented. We reformulate the equation as a feasibility problem which we solve via the method of alternating projections (MAP). The fixed points set of the alternating projections map is characterized under nondegeneracy conditions on $A$ and $B$. Furtherm… ▽ More A novel approach for solving the general absolute value equation $Ax+B|x| = c$ where $A,B\in \mathbb{R}^{m\times n}$ and $c\in \mathbb{R}^m$ is presented. We reformulate the equation as a feasibility problem which we solve via the method of alternating projections (MAP). The fixed points set of the alternating projections map is characterized under nondegeneracy conditions on $A$ and $B$. Furthermore, we prove linear convergence of the algorithm. Unlike most of the existing approaches in the literature, the algorithm presented here is capable of handling problems with $m\neq n$, both theoretically and numerically. △ Less

Submitted 6 June, 2021; originally announced June 2021.

Comments: 38 pages, 1 figure, 3 tables

Strengthened Splitting Methods for Computing Resolvents

Authors: Francisco J. Aragón Artacho, Rubén Campoy, Matthew K. Tam

Abstract: In this work, we develop a systematic framework for computing the resolvent of the sum of two or more monotone operators which only activates each operator in the sum individually. The key tool in the development of this framework is the notion of the "strengthening" of a set-valued operator, which can be viewed as a type of regularisation that preserves computational tractability. After deriving… ▽ More In this work, we develop a systematic framework for computing the resolvent of the sum of two or more monotone operators which only activates each operator in the sum individually. The key tool in the development of this framework is the notion of the "strengthening" of a set-valued operator, which can be viewed as a type of regularisation that preserves computational tractability. After deriving a number of iterative schemes through this framework, we demonstrate their application to best approximation problems, image denoising and elliptic PDEs. △ Less

Submitted 10 June, 2021; v1 submitted 3 November, 2020; originally announced November 2020.

MSC Class: 47H05; 90C30; 65K05

Gearhart-Koshy Acceleration for Affine Subspaces

Authors: Matthew K. Tam

Abstract: The method of cyclic projections finds nearest points in the intersection of finitely many affine subspaces. To accelerate convergence, Gearhart and Koshy proposed a modification which, in each iteration, performs an exact line search based on minimising the distance to the solution. When the subspaces are linear, the procedure can be made explicit using feasibility of the zero vector. This work s… ▽ More The method of cyclic projections finds nearest points in the intersection of finitely many affine subspaces. To accelerate convergence, Gearhart and Koshy proposed a modification which, in each iteration, performs an exact line search based on minimising the distance to the solution. When the subspaces are linear, the procedure can be made explicit using feasibility of the zero vector. This work studies an alternative approach which does not rely on this fact, thus providing an efficient implementation in the affine setting. △ Less

Submitted 8 December, 2020; v1 submitted 1 July, 2020; originally announced July 2020.

Comments: 8 pages, 1 figure

A Douglas-Rachford construction of non-separable continuous compactly supported multidimensional wavelets

Authors: David Franklin, Jeffrey A. Hogan, Matthew K. Tam

Abstract: After re-casting the $n$-dimensional wavelet construction problem as a feasibility problem with constraints arising from the requirements of compact support, smoothness and orthogonality, the Douglas--Rachford algorithm is employed in the search for one- and two-dimensional wavelets. New one-dimensional wavelets are produced as well as genuinely non-separable two-dimensional wavelets in the case w… ▽ More After re-casting the $n$-dimensional wavelet construction problem as a feasibility problem with constraints arising from the requirements of compact support, smoothness and orthogonality, the Douglas--Rachford algorithm is employed in the search for one- and two-dimensional wavelets. New one-dimensional wavelets are produced as well as genuinely non-separable two-dimensional wavelets in the case where the dilation on the plane is the standard $D_af(t)=a^{-1}f(t/a)$ $(t\in{\mathbb R}^n, a>0)$. △ Less

Submitted 5 June, 2020; originally announced June 2020.

Comments: 36 pages, 4 figures

MSC Class: 42C40 (Primary) 42B99; 65T60; 47N10; 65K10; 65T60 (Secondary)

Convergence Rates for Boundedly Regular Systems

Authors: Ernö Robert Csetnek, Andrew Eberhard, Matthew K. Tam

Abstract: In this work, we consider a continuous dynamical system associated with the fixed point set of a nonexpansive operator which was originally studied by Boţ & Csetnek (2015). Our main results establish convergence rates for the system's trajectories when the nonexpansive operator satisfies an additional regularity property. This setting is the natural continuous-time analogue to discrete-time result… ▽ More In this work, we consider a continuous dynamical system associated with the fixed point set of a nonexpansive operator which was originally studied by Boţ & Csetnek (2015). Our main results establish convergence rates for the system's trajectories when the nonexpansive operator satisfies an additional regularity property. This setting is the natural continuous-time analogue to discrete-time results obtained in Bauschke, Noll & Phan (2015) and Borwein, Li & Tam (2017) by using the same regularity properties. △ Less

Submitted 7 June, 2021; v1 submitted 2 April, 2020; originally announced April 2020.

Comments: 15 pages

Backward-Forward-Reflected-Backward Splitting for Three Operator Monotone Inclusions

Authors: Janosch Rieger, Matthew K. Tam

Abstract: In this work, we propose and analyse two splitting algorithms for finding a zero of the sum of three monotone operators, one of which is assumed to be Lipschitz continuous. Each iteration of these algorithms require one forward evaluation of the Lipschitz continuous operator and one resolvent evaluation of each of the other two operators. By specialising to two operator inclusions, we recover the… ▽ More In this work, we propose and analyse two splitting algorithms for finding a zero of the sum of three monotone operators, one of which is assumed to be Lipschitz continuous. Each iteration of these algorithms require one forward evaluation of the Lipschitz continuous operator and one resolvent evaluation of each of the other two operators. By specialising to two operator inclusions, we recover the forward-reflected-backward and the reflected-forward-backward splitting methods as particular cases. The inspiration for the proposed algorithms arises from interpretations of the aforementioned reflected splitting algorithms as discretisations of the continuous-time proximal point algorithm. △ Less

Submitted 20 January, 2020; originally announced January 2020.

Comments: 15 pages

The Douglas-Rachford Algorithm for Convex and Nonconvex Feasibility Problems

Authors: Francisco J. Aragón Artacho, Rubén Campoy, Matthew K. Tam

Abstract: The Douglas-Rachford method, a projection algorithm designed to solve continuous optimization problems, forms the basis of a useful heuristic for solving combinatorial optimization problems. In order to successfully use the method, it is necessary to formulate the problem at hand as a feasibility problem with constraint sets having efficiently computable nearest points. In this self-contained tuto… ▽ More The Douglas-Rachford method, a projection algorithm designed to solve continuous optimization problems, forms the basis of a useful heuristic for solving combinatorial optimization problems. In order to successfully use the method, it is necessary to formulate the problem at hand as a feasibility problem with constraint sets having efficiently computable nearest points. In this self-contained tutorial, we develop the convergence theory of projection algorithms within the framework of fixed point iterations, explain how to devise useful feasibility problem formulations, and demonstrate the application of the Douglas-Rachford method to said formulations. The paradigm is then illustrated on two concrete problems: a generalization of the "eight queens puzzle" known as the "(m,n)-queens problem", and the problem of constructing a probability distribution with prescribed moments. △ Less

Submitted 19 April, 2019; originally announced April 2019.

MSC Class: 65K05; 90C27; 90-01; 65-01

Shadow Douglas--Rachford Splitting for Monotone Inclusions

Authors: Ernö Robert Csetnek, Yura Malitsky, Matthew K. Tam

Abstract: In this work, we propose a new algorithm for finding a zero in the sum of two monotone operators where one is assumed to be single-valued and Lipschitz continuous. This algorithm naturally arises from a non-standard discretization of a continuous dynamical system associated with the Douglas--Rachford splitting algorithm. More precisely, it is obtained by performing an explicit, rather than implici… ▽ More In this work, we propose a new algorithm for finding a zero in the sum of two monotone operators where one is assumed to be single-valued and Lipschitz continuous. This algorithm naturally arises from a non-standard discretization of a continuous dynamical system associated with the Douglas--Rachford splitting algorithm. More precisely, it is obtained by performing an explicit, rather than implicit, discretization with respect to one of the operators involved. Each iteration of the proposed algorithm requires the evaluation of one forward and one backward operator. △ Less

Submitted 8 March, 2019; originally announced March 2019.

MSC Class: 49M29; 90C25; 47H05; 47J20; 65K15

Journal ref: Applied Mathematics & Optimization 80, 665-678 (2019)

Characterizations of Super-regularity and its Variants

Authors: Aris Daniilidis, D. Russell Luke, Matthew K. Tam

Abstract: Convergence of projection-based methods for nonconvex set feasibility problems has been established for sets with ever weaker regularity assumptions. What has not kept pace with these developments is analogous results for convergence of optimization problems with correspondingly weak assumptions on the value functions. Indeed, one of the earliest classes of nonconvex sets for which convergence res… ▽ More Convergence of projection-based methods for nonconvex set feasibility problems has been established for sets with ever weaker regularity assumptions. What has not kept pace with these developments is analogous results for convergence of optimization problems with correspondingly weak assumptions on the value functions. Indeed, one of the earliest classes of nonconvex sets for which convergence results were obtainable, the class of so-called super-regular sets introduced by Lewis, Luke and Malick (2009), has no functional counterpart. In this work, we amend this gap in the theory by establishing the equivalence between a property slightly stronger than super-regularity, which we call Clarke super-regularity, and subsmootheness of sets as introduced by Aussel, Daniilidis and Thibault (2004). The bridge to functions shows that approximately convex functions studied by Ngai, Luc and Théra (2000) are those which have Clarke super-regular epigraphs. Further classes of regularity of functions based on the corresponding regularity of their epigraph are also discussed. △ Less

Submitted 15 August, 2018; originally announced August 2018.

Comments: 15 pages, 2 figures

Journal ref: Splitting Algorithms, Modern Operator Theory and Applications (2019)

A Forward-Backward Splitting Method for Monotone Inclusions Without Cocoercivity

Authors: Yura Malitsky, Matthew K. Tam

Abstract: In this work, we propose a simple modification of the forward-backward splitting method for finding a zero in the sum of two monotone operators. Our method converges under the same assumptions as Tseng's forward-backward-forward method, namely, it does not require cocoercivity of the single-valued operator. Moreover, each iteration only requires one forward evaluation rather than two as is the cas… ▽ More In this work, we propose a simple modification of the forward-backward splitting method for finding a zero in the sum of two monotone operators. Our method converges under the same assumptions as Tseng's forward-backward-forward method, namely, it does not require cocoercivity of the single-valued operator. Moreover, each iteration only requires one forward evaluation rather than two as is the case for Tseng's method. Variants of the method incorporating a linesearch, relaxation and inertia, or a structured three operator inclusion are also discussed. △ Less

Submitted 6 May, 2020; v1 submitted 13 August, 2018; originally announced August 2018.

Comments: 20 pages, 1 figure

MSC Class: 49M2; 90C25; 47H05; 47J20; 65K15

Union Averaged Operators with Applications to Proximal Algorithms for Min-Convex Functions

Authors: Minh N. Dao, Matthew K. Tam

Abstract: In this paper we introduce and study a class of structured set-valued operators which we call union averaged nonexpansive. At each point in their domain, the value of such an operator can be expressed as a finite union of single-valued averaged nonexpansive operators. We investigate various structural properties of the class and show, in particular, that is closed under taking unions, convex combi… ▽ More In this paper we introduce and study a class of structured set-valued operators which we call union averaged nonexpansive. At each point in their domain, the value of such an operator can be expressed as a finite union of single-valued averaged nonexpansive operators. We investigate various structural properties of the class and show, in particular, that is closed under taking unions, convex combinations, and compositions, and that their fixed point iterations are locally convergent around strong fixed points. We then systematically apply our results to analyze proximal algorithms in situations where union averaged nonexpansive operators naturally arise. In particular, we consider the problem of minimizing the sum two functions where the first is convex and the second can be expressed as the minimum of finitely many convex functions. △ Less

Submitted 17 October, 2018; v1 submitted 16 July, 2018; originally announced July 2018.

Journal ref: Journal of Optimization Theory and Applications, 181:61-94, 2019

A feasibility approach for constructing combinatorial designs of circulant type

Authors: Francisco J. Aragón Artacho, Rubén Campoy, Ilias Kotsireas, Matthew K. Tam

Abstract: In this work, we propose an optimization approach for constructing various classes of circulant combinatorial designs that can be defined in terms of autocorrelations. The problem is formulated as a so-called feasibility problem having three sets, to which the Douglas-Rachford projection algorithm is applied. The approach is illustrated on three different classes of circulant combinatorial designs… ▽ More In this work, we propose an optimization approach for constructing various classes of circulant combinatorial designs that can be defined in terms of autocorrelations. The problem is formulated as a so-called feasibility problem having three sets, to which the Douglas-Rachford projection algorithm is applied. The approach is illustrated on three different classes of circulant combinatorial designs: circulant weighing matrices, D-optimal matrices, and Hadamard matrices with two circulant cores. Furthermore, we explicitly construct two new circulant weighing matrices, a $CW(126,64)$ and a $CW(198,100)$, whose existence was previously marked as unresolved in the most recent version of Strassler's table. △ Less

Submitted 7 November, 2017; originally announced November 2017.

MSC Class: 05B20; 90C59; 47J25; 47N10

Journal ref: Journal of Combinatoral Optimization, 35(4):1061-1085, 2018

Algorithms Based on Unions of Nonexpansive Maps

Authors: Matthew K. Tam

Abstract: In this note, we consider a framework for the analysis of iterative algorithms which can described in terms of a structured set-valued operator. More precisely, at each point in the ambient space, we assume that the value of operator can be expressed as a finite union of values of single-valued paracontracting operators. Our main result, which shows that the associated fixed point iteration is loc… ▽ More In this note, we consider a framework for the analysis of iterative algorithms which can described in terms of a structured set-valued operator. More precisely, at each point in the ambient space, we assume that the value of operator can be expressed as a finite union of values of single-valued paracontracting operators. Our main result, which shows that the associated fixed point iteration is locally convergent around strong fixed points, generalises a theorem due to Bauschke and Noll (2014). △ Less

Submitted 23 February, 2018; v1 submitted 23 October, 2017; originally announced October 2017.

Comments: 8 pages

Journal ref: Optimization Letters 12(5):1019-1027, 2018

A Lyapunov-type approach to convergence of the Douglas-Rachford algorithm

Authors: Minh N. Dao, Matthew K. Tam

Abstract: The Douglas-Rachford projection algorithm is an iterative method used to find a point in the intersection of closed constraint sets. The algorithm has been experimentally observed to solve various nonconvex feasibility problems which current theory cannot sufficiently explain. In this paper, we prove convergence of the Douglas-Rachford algorithm in a potentially nonconvex setting. Our analysis rel… ▽ More The Douglas-Rachford projection algorithm is an iterative method used to find a point in the intersection of closed constraint sets. The algorithm has been experimentally observed to solve various nonconvex feasibility problems which current theory cannot sufficiently explain. In this paper, we prove convergence of the Douglas-Rachford algorithm in a potentially nonconvex setting. Our analysis relies on the existence of a Lyapunov-type functional whose convexity properties are not tantamount to convexity of the original constraint sets. Moreover, we provide various nonconvex examples in which our framework proves global convergence of the algorithm. △ Less

Submitted 15 June, 2017; originally announced June 2017.

Comments: 31 pages, 4 figures

Journal ref: Journal of Global Optimization, 73:83-112, 2019

Symbolic computation with monotone operators

Authors: Florian Lauster, D. Russell Luke, Matthew K. Tam

Abstract: We consider a class of monotone operators which are appropriate for symbolic representation and manipulation within a computer algebra system. Various structural properties of the class (e.g., closure under taking inverses, resolvents) are investigated as well as the role played by maximal monotonicity within the class. In particular, we show that there is a natural correspondence between our clas… ▽ More We consider a class of monotone operators which are appropriate for symbolic representation and manipulation within a computer algebra system. Various structural properties of the class (e.g., closure under taking inverses, resolvents) are investigated as well as the role played by maximal monotonicity within the class. In particular, we show that there is a natural correspondence between our class of monotone operators and the subdifferentials of convex functions belonging to a class of convex functions deemed suitable for symbolic computation of Fenchel conjugates which were previously studied by Bauschke & von Mohrenschildt and by Borwein & Hamilton. A number of illustrative examples utilizing the introduced class of operators are provided including computation of proximity operators, recovery of a convex penalty function associated with the hard thresholding operator, and computation of superexpectations, superdistributions and superquantiles with specialization to risk measures. △ Less

Submitted 17 March, 2017; originally announced March 2017.

Comments: 17 pages, 2 figures

MSC Class: 47H05; 47N10; 68W30

Journal ref: Set-Valued and Variational Analysis 26, 2018

Regularity Properties of Non-Negative Sparsity Sets

Authors: Matthew K. Tam

Abstract: This paper investigates regularity properties of two non-negative sparsity sets: non-negative sparse vectors, and low-rank positive semi-definite matrices. Novel formulae for their Mordukhovich normal cones are given and used to formulate sufficient conditions for non-convex notions of regularity to hold. Our results provide a useful tool for justifying the application of projection methods to cer… ▽ More This paper investigates regularity properties of two non-negative sparsity sets: non-negative sparse vectors, and low-rank positive semi-definite matrices. Novel formulae for their Mordukhovich normal cones are given and used to formulate sufficient conditions for non-convex notions of regularity to hold. Our results provide a useful tool for justifying the application of projection methods to certain rank constrained feasibility problems. △ Less

Submitted 20 July, 2016; originally announced July 2016.

Comments: 22 pages

Journal ref: Journal of Mathematical Analysis and Applications, 447(2), 2017

Quantitative convergence analysis of iterated expansive, set-valued mappings

Authors: D. Russell Luke, Nguyen H. Thao, Matthew K. Tam

Abstract: We develop a framework for quantitative convergence analysis of Picard iterations of expansive set-valued fixed point mappings. There are two key components of the analysis. The first is a natural generalization of single-valued averaged mappings to expansive, set-valued mappings that characterizes a type of strong calmness of the fixed point mapping. The second component to this analysis is an ex… ▽ More We develop a framework for quantitative convergence analysis of Picard iterations of expansive set-valued fixed point mappings. There are two key components of the analysis. The first is a natural generalization of single-valued averaged mappings to expansive, set-valued mappings that characterizes a type of strong calmness of the fixed point mapping. The second component to this analysis is an extension of the well-established notion of metric subregularity -- or inverse calmness -- of the mapping at fixed points. Convergence of expansive fixed point iterations is proved using these two properties, and quantitative estimates are a natural byproduct of the framework. To demonstrate the application of the theory, we prove for the first time a number of results showing local linear convergence of nonconvex cyclic projections for inconsistent (and consistent) feasibility problems, local linear convergence of the forward-backward algorithm for structured optimization without convexity, strong or otherwise, and local linear convergence of the Douglas--Rachford algorithm for structured nonconvex minimization. This theory includes earlier approaches for known results, convex and nonconvex, as special cases. △ Less

Submitted 25 March, 2017; v1 submitted 18 May, 2016; originally announced May 2016.

Comments: 41 pages 70 references. Detailed examples added and more historical commentary (with citations)

MSC Class: 49J53; 65K10; 49K40; 49M05; 49M27; 65K05; 90C26

Journal ref: Mathematics of Operations Research, online 2018

Convergence rate analysis for averaged fixed point iterations in the presence of Hölder regularity

Authors: Jonathan M. Borwein, Guoyin Li, Matthew K. Tam

Abstract: In this paper, we establish sublinear and linear convergence of fixed point iterations generated by averaged operators in a Hilbert space. Our results are achieved under a bounded Hölder regularity assumption which generalizes the well-known notion of bounded linear regularity. As an application of our results, we provide a convergence rate analysis for Krasnoselskii-Mann iterations, the cyclic pr… ▽ More In this paper, we establish sublinear and linear convergence of fixed point iterations generated by averaged operators in a Hilbert space. Our results are achieved under a bounded Hölder regularity assumption which generalizes the well-known notion of bounded linear regularity. As an application of our results, we provide a convergence rate analysis for Krasnoselskii-Mann iterations, the cyclic projection algorithm, and the Douglas-Rachford feasibility algorithm along with some variants. In the important case in which the underlying sets are convex sets described by convex polynomials in a finite dimensional space, we show that the Hölder regularity properties are automatically satisfied, from which sublinear convergence follows. △ Less

Submitted 18 August, 2016; v1 submitted 23 October, 2015; originally announced October 2015.

Comments: 34 pages, 1 figure

Journal ref: SIAM J. Optim., 27(1):1-33, 2017

Global Behavior of the Douglas-Rachford Method for a Nonconvex Feasibility Problem

Authors: Francisco J. Aragón Artacho, Jonathan M. Borwein, Matthew K. Tam

Abstract: In recent times the Douglas-Rachford algorithm has been observed empirically to solve a variety of nonconvex feasibility problems including those of a combinatorial nature. For many of these problems current theory is not sufficient to explain this observed success and is mainly concerned with questions of local convergence. In this paper we analyze global behavior of the method for finding a poin… ▽ More In recent times the Douglas-Rachford algorithm has been observed empirically to solve a variety of nonconvex feasibility problems including those of a combinatorial nature. For many of these problems current theory is not sufficient to explain this observed success and is mainly concerned with questions of local convergence. In this paper we analyze global behavior of the method for finding a point in the intersection of a half-space and a potentially non-convex set which is assumed to satisfy a well-quasi-ordering property or a property weaker than compactness. In particular, the special case in which the second set is finite is covered by our framework and provides a prototypical setting for combinatorial optimization problems. △ Less

Submitted 30 June, 2015; originally announced June 2015.

MSC Class: 90C26; 65K05

Journal ref: Journal of Global Optimization 65 (2016), 309-327

Reflection methods for inverse problems with application to protein conformation determination

Authors: Jonathan M. Borwein, Matthew K. Tam

Abstract: The Douglas-Rachford reflection method is a general purpose algorithm useful for solving the feasibility problem of finding a point in the intersection of finitely many sets. In this chapter we demonstrate that applied to a specific problem, the method can benefit from heuristics specific to said problem which exploit its special structure. In particular, we focus on the problem of protein conform… ▽ More The Douglas-Rachford reflection method is a general purpose algorithm useful for solving the feasibility problem of finding a point in the intersection of finitely many sets. In this chapter we demonstrate that applied to a specific problem, the method can benefit from heuristics specific to said problem which exploit its special structure. In particular, we focus on the problem of protein conformation determination formulated within the framework of matrix completion, as was considered in a recent paper of the present authors. △ Less

Submitted 19 August, 2014; originally announced August 2014.

Comments: 18 pages, 8 figures

Journal ref: In: Aussel D., Lalitha C. (eds) Generalized Nash Equilibrium Problems, Bilevel Programming and MPEC (2017)

Proximal Heterogeneous Block Input-Output Method and application to Blind Ptychographic Diffraction Imaging

Authors: Robert Hesse, D. Russell Luke, Shoham Sabach, Matthew K. Tam

Abstract: We propose a general alternating minimization algorithm for nonconvex optimization problems with separable structure and nonconvex coupling between blocks of variables. To fix our ideas, we apply the methodology to the problem of blind ptychographic imaging. Compared to other schemes in the literature, our approach differs in two ways: (i) it is posed within a clear mathematical framework with pra… ▽ More We propose a general alternating minimization algorithm for nonconvex optimization problems with separable structure and nonconvex coupling between blocks of variables. To fix our ideas, we apply the methodology to the problem of blind ptychographic imaging. Compared to other schemes in the literature, our approach differs in two ways: (i) it is posed within a clear mathematical framework with practically verifiable assumptions, and (ii) under the given assumptions, it is provably convergent to critical points. A numerical comparison of our proposed algorithm with the current state-of-the-art on simulated and experimental data validates our approach and points toward directions for further improvement. △ Less

Submitted 8 August, 2014; originally announced August 2014.

Comments: 32 pages, 3 tables, 5 figures

MSC Class: 90C05; 90C25; 90C30; 90C52; 65K05

Journal ref: SIAM J. on Imaging Sciences, 8(1):426--457 (2015)

Norm Convergence of Realistic Projection and Reflection Methods

Authors: Jonathan M. Borwein, Brailey Sims, Matthew K. Tam

Abstract: We provide sufficient conditions for norm convergence of various projection and reflection methods, as well as giving limiting examples regarding convergence rates. We provide sufficient conditions for norm convergence of various projection and reflection methods, as well as giving limiting examples regarding convergence rates. △ Less

Submitted 2 July, 2014; v1 submitted 27 December, 2013; originally announced December 2013.

Comments: 21 pages, 1 figure, references updated

The Cyclic Douglas-Rachford Method for Inconsistent Feasibility Problems

Authors: Jonathan M. Borwein, Matthew K. Tam

Abstract: We analyse the behaviour of the newly introduced cyclic Douglas-Rachford algorithm for finding a point in the intersection of a finite number of closed convex sets. This work considers the case in which the target intersection set is possibly empty. We analyse the behaviour of the newly introduced cyclic Douglas-Rachford algorithm for finding a point in the intersection of a finite number of closed convex sets. This work considers the case in which the target intersection set is possibly empty. △ Less

Submitted 2 July, 2014; v1 submitted 8 October, 2013; originally announced October 2013.

Comments: 13 pages, 2 figures; references updated, figure 2 corrected

Journal ref: Journal of Nonlinear and Convex Analysis 16(4), 2015

Douglas-Rachford Feasibility Methods for Matrix Completion Problems

Authors: Francisco J. Aragón Artacho, Jonathan M. Borwein, Matthew K. Tam

Abstract: In this paper we give general recommendations for successful application of the Douglas-Rachford reflection method to convex and non-convex real matrix-completion problems. These guidelines are demonstrated by various illustrative examples. In this paper we give general recommendations for successful application of the Douglas-Rachford reflection method to convex and non-convex real matrix-completion problems. These guidelines are demonstrated by various illustrative examples. △ Less

Submitted 20 August, 2013; originally announced August 2013.

Comments: 30 pages, 4 figures, 3 tables

Recent Results on Douglas-Rachford Methods for Combinatorial Optimization Problems

Authors: Francisco J. Aragón Artacho, Jonathan M. Borwein, Matthew K. Tam

Abstract: We discuss recent positive experiences applying convex feasibility algorithms of Douglas--Rachford type to highly combinatorial and far from convex problems. We discuss recent positive experiences applying convex feasibility algorithms of Douglas--Rachford type to highly combinatorial and far from convex problems. △ Less

Submitted 12 May, 2013; originally announced May 2013.

MSC Class: 90C27; 90C59; 47N10

Journal ref: Journal of Optimization Theory and Applications 163 (2014), 1-30

A Cyclic Douglas-Rachford Iteration Scheme

Authors: Jonathan M. Borwein, Matthew K. Tam

Abstract: In this paper we present two Douglas-Rachford inspired iteration schemes which can be applied directly to N-set convex feasibility problems in Hilbert space. Our main results are weak convergence of the methods to a point whose nearest point projections onto each of the N sets coincide. For affine subspaces, convergence is in norm. Initial results from numerical experiments, comparing our methods… ▽ More In this paper we present two Douglas-Rachford inspired iteration schemes which can be applied directly to N-set convex feasibility problems in Hilbert space. Our main results are weak convergence of the methods to a point whose nearest point projections onto each of the N sets coincide. For affine subspaces, convergence is in norm. Initial results from numerical experiments, comparing our methods to the classical (product-space) Douglas-Rachford scheme, are promising. △ Less

Submitted 10 April, 2013; v1 submitted 7 March, 2013; originally announced March 2013.

Comments: 22 pages, 7 figures, 4 tables

Journal ref: Journal of Optimization Theory and Applications 160(1), 2014