Showing 1–7 of 7 results for author: Maragos, P

Revisiting Tropical Polynomial Division: Theory, Algorithms and Application to Neural Networks

Authors: Ioannis Kordonis, Petros Maragos

Abstract: Tropical geometry has recently found several applications in the analysis of neural networks with piecewise linear activation functions. This paper presents a new look at the problem of tropical polynomial division and its application to the simplification of neural networks. We analyze tropical polynomials with real coefficients, extending earlier ideas and methods developed for polynomials with… ▽ More Tropical geometry has recently found several applications in the analysis of neural networks with piecewise linear activation functions. This paper presents a new look at the problem of tropical polynomial division and its application to the simplification of neural networks. We analyze tropical polynomials with real coefficients, extending earlier ideas and methods developed for polynomials with integer coefficients. We first prove the existence of a unique quotient-remainder pair and characterize the quotient in terms of the convex bi-conjugate of a related function. Interestingly, the quotient of tropical polynomials with integer coefficients does not necessarily have integer coefficients. Furthermore, we develop a relationship of tropical polynomial division with the computation of the convex hull of unions of convex polyhedra and use it to derive an exact algorithm for tropical polynomial division. An approximate algorithm is also presented, based on an alternation between data partition and linear programming. We also develop special techniques to divide composite polynomials, described as sums or maxima of simpler ones. Finally, we present some numerical results to illustrate the efficiency of the algorithms proposed, using the MNIST handwritten digit and CIFAR-10 datasets. △ Less

Submitted 26 June, 2023; originally announced June 2023.

Sparse Approximate Solutions to Max-Plus Equations with Application to Multivariate Convex Regression

Authors: Nikos Tsilivis, Anastasios Tsiamis, Petros Maragos

Abstract: In this work, we study the problem of finding approximate, with minimum support set, solutions to matrix max-plus equations, which we call sparse approximate solutions. We show how one can obtain such solutions efficiently and in polynomial time for any $\ell_p$ approximation error. Based on these results, we propose a novel method for piecewise-linear fitting of convex multivariate functions, wit… ▽ More In this work, we study the problem of finding approximate, with minimum support set, solutions to matrix max-plus equations, which we call sparse approximate solutions. We show how one can obtain such solutions efficiently and in polynomial time for any $\ell_p$ approximation error. Based on these results, we propose a novel method for piecewise-linear fitting of convex multivariate functions, with optimality guarantees for the model parameters and an approximately minimum number of affine regions. △ Less

Submitted 21 December, 2020; v1 submitted 6 November, 2020; originally announced November 2020.

Comments: 20 pages, 5 figures, 5 tables. Introduction revision and typos correction

Tropical Geometry and Piecewise-Linear Approximation of Curves and Surfaces on Weighted Lattices

Authors: Petros Maragos, Emmanouil Theodosis

Abstract: Tropical Geometry and Mathematical Morphology share the same max-plus and min-plus semiring arithmetic and matrix algebra. In this chapter we summarize some of their main ideas and common (geometric and algebraic) structure, generalize and extend both of them using weighted lattices and a max-$\star$ algebra with an arbitrary binary operation $\star$ that distributes over max, and outline applicat… ▽ More Tropical Geometry and Mathematical Morphology share the same max-plus and min-plus semiring arithmetic and matrix algebra. In this chapter we summarize some of their main ideas and common (geometric and algebraic) structure, generalize and extend both of them using weighted lattices and a max-$\star$ algebra with an arbitrary binary operation $\star$ that distributes over max, and outline applications to geometry, machine learning, and optimization. Further, we generalize tropical geometrical objects using weighted lattices. Finally, we provide the optimal solution of max-$\star$ equations using morphological adjunctions that are projections on weighted lattices, and apply it to optimal piecewise-linear regression for fitting max-$\star$ tropical curves and surfaces to arbitrary data that constitute polygonal or polyhedral shape approximations. This also includes an efficient algorithm for solving the convex regression problem of data fitting with max-affine functions. △ Less

Submitted 9 December, 2019; originally announced December 2019.

Comments: 39 pages, 13 figures

Tropical Polynomial Division and Neural Networks

Authors: Georgios Smyrnis, Petros Maragos

Abstract: In this work, we examine the process of Tropical Polynomial Division, a geometric method which seeks to emulate the division of regular polynomials, when applied to those of the max-plus semiring. This is done via the approximation of the Newton Polytope of the dividend polynomial by that of the divisor. This process is afterwards generalized and applied in the context of neural networks with ReLU… ▽ More In this work, we examine the process of Tropical Polynomial Division, a geometric method which seeks to emulate the division of regular polynomials, when applied to those of the max-plus semiring. This is done via the approximation of the Newton Polytope of the dividend polynomial by that of the divisor. This process is afterwards generalized and applied in the context of neural networks with ReLU activations. In particular, we make use of the intuition it provides, in order to minimize a two-layer fully connected network, trained for a binary classification problem. This method is later evaluated on a variety of experiments, demonstrating its capability to approximate a network, with minimal loss in performance. △ Less

Submitted 28 November, 2019; originally announced November 2019.

Tropical Modeling of Weighted Transducer Algorithms on Graphs

Authors: Emmanouil Theodosis, Petros Maragos

Abstract: Weighted Finite State Transducers (WFSTs) are versatile data structures that can model a great number of problems, ranging from Automatic Speech Recognition to DNA sequencing. Traditional computer science algorithms are employed when working with these structures in order to optimise their size, but also the runtime of decoding algorithms. However, these algorithms are not unified under a common f… ▽ More Weighted Finite State Transducers (WFSTs) are versatile data structures that can model a great number of problems, ranging from Automatic Speech Recognition to DNA sequencing. Traditional computer science algorithms are employed when working with these structures in order to optimise their size, but also the runtime of decoding algorithms. However, these algorithms are not unified under a common framework that would allow for their treatment as a whole. Moreover, the inherent geometrical representation of WFSTs, coupled with the topology-preserving algorithms that operate on them make the structures ideal for tropical analysis. The benefits of such analysis have a twofold nature; first, matrix operations offer a connection to nonlinear vector space and spectral theory, and, second, tropical algebra offers a connection to tropical geometry. In this work we model some of the most frequently used algorithms in WFSTs by using tropical algebra; this provides a theoretical unification and allows us to also analyse aspects of their tropical geometry. Further, we provide insights via numerical examples. △ Less

Submitted 1 November, 2018; originally announced November 2018.

Comments: Under review for the International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

Sparsity in Max-Plus Algebra and Systems

Authors: Anastasios Tsiamis, Petros Maragos

Abstract: We study sparsity in the max-plus algebraic setting. We seek both exact and approximate solutions of the max-plus linear equation with minimum cardinality of support. In the former case, the sparsest solution problem is shown to be equivalent to the minimum set cover problem and, thus, NP-complete. In the latter one, the approximation is quantified by the $\ell_{1}$ residual error norm, which is s… ▽ More We study sparsity in the max-plus algebraic setting. We seek both exact and approximate solutions of the max-plus linear equation with minimum cardinality of support. In the former case, the sparsest solution problem is shown to be equivalent to the minimum set cover problem and, thus, NP-complete. In the latter one, the approximation is quantified by the $\ell_{1}$ residual error norm, which is shown to have supermodular properties under some convex constraints, called lateness constraints. Thus, greedy approximation algorithms of polynomial complexity can be employed for both problems with guaranteed bounds of approximation. We also study the sparse recovery problem and present conditions, under which, the sparsest exact solution solves it. Through multi-machine interactive processes, we describe how the present framework could be applied to two practical discrete event systems problems: resource optimization and structure-seeking system identification. We also show how sparsity is related to the pruning problem. Finally, we present a numerical example of the structure-seeking system identification problem and we study the performance of the greedy algorithm via simulations. △ Less

Submitted 4 June, 2019; v1 submitted 29 January, 2018; originally announced January 2018.

Comments: The paper was published in Discrete Event Dynamic Systems journal

Stochastic Stability in Max-Product and Max-Plus Systems with Markovian Jumps

Authors: Ioannis Kordonis, Petros Maragos, George P. Papavassilopoulos

Abstract: We study Max-Product and Max-Plus Systems with Markovian Jumps and focus on stochastic stability problems. At first, a Lyapunov function is derived for the asymptotically stable deterministic Max-Product Systems. This Lyapunov function is then adjusted to derive sufficient conditions for the stochastic stability of Max-Product systems with Markovian Jumps. Many step Lyapunov functions are then use… ▽ More We study Max-Product and Max-Plus Systems with Markovian Jumps and focus on stochastic stability problems. At first, a Lyapunov function is derived for the asymptotically stable deterministic Max-Product Systems. This Lyapunov function is then adjusted to derive sufficient conditions for the stochastic stability of Max-Product systems with Markovian Jumps. Many step Lyapunov functions are then used to derive necessary and sufficient conditions for stochastic stability. The results for the Max-Product systems are then applied to Max-Plus systems with Markovian Jumps, using an isomorphism and almost sure bounds for the asymptotic behavior of the state are obtained. A numerical example illustrating the application of the stability results on a production system is also given. △ Less

Submitted 8 November, 2017; originally announced November 2017.