-
Mirror Descent on Reproducing Kernel Banach Spaces
Authors:
Akash Kumar,
Mikhail Belkin,
Parthe Pandit
Abstract:
Recent advances in machine learning have led to increased interest in reproducing kernel Banach spaces (RKBS) as a more general framework that extends beyond reproducing kernel Hilbert spaces (RKHS). These works have resulted in the formulation of representer theorems under several regularized learning schemes. However, little is known about an optimization method that encompasses these results in…
▽ More
Recent advances in machine learning have led to increased interest in reproducing kernel Banach spaces (RKBS) as a more general framework that extends beyond reproducing kernel Hilbert spaces (RKHS). These works have resulted in the formulation of representer theorems under several regularized learning schemes. However, little is known about an optimization method that encompasses these results in this setting. This paper addresses a learning problem on Banach spaces endowed with a reproducing kernel, focusing on efficient optimization within RKBS. To tackle this challenge, we propose an algorithm based on mirror descent (MDA). Our approach involves an iterative method that employs gradient steps in the dual space of the Banach space using the reproducing kernel.
We analyze the convergence properties of our algorithm under various assumptions and establish two types of results: first, we identify conditions under which a linear convergence rate is achievable, akin to optimization in the Euclidean setting, and provide a proof of the linear rate; second, we demonstrate a standard convergence rate in a constrained setting. Moreover, to instantiate this algorithm in practice, we introduce a novel family of RKBSs with $p$-norm ($p \neq 2$), characterized by both an explicit dual map and a kernel.
△ Less
Submitted 17 November, 2024;
originally announced November 2024.
-
Eigenvectors of the De Bruijn Graph Laplacian: A Natural Basis for the Cut and Cycle Space
Authors:
Anthony Philippakis,
Neil Mallinar,
Parthe Pandit,
Mikhail Belkin
Abstract:
We study the Laplacian of the undirected De Bruijn graph over an alphabet $A$ of order $k$. While the eigenvalues of this Laplacian were found in 1998 by Delorme and Tillich [1], an explicit description of its eigenvectors has remained elusive. In this work, we find these eigenvectors in closed form and show that they yield a natural and canonical basis for the cut- and cycle-spaces of De Bruijn g…
▽ More
We study the Laplacian of the undirected De Bruijn graph over an alphabet $A$ of order $k$. While the eigenvalues of this Laplacian were found in 1998 by Delorme and Tillich [1], an explicit description of its eigenvectors has remained elusive. In this work, we find these eigenvectors in closed form and show that they yield a natural and canonical basis for the cut- and cycle-spaces of De Bruijn graphs. Remarkably, we find that the cycle basis we construct is a basis for the cycle space of both the undirected and the directed De Bruijn graph. This is done by developing an analogue of the Fourier transform on the De Bruijn graph, which acts to diagonalize the Laplacian. Moreover, we show that the cycle-space of De Bruijn graphs, when considering all possible orders of $k$ simultaneously, contains a rich algebraic structure, that of a graded Hopf algebra.
△ Less
Submitted 10 October, 2024;
originally announced October 2024.
-
Universality of kernel random matrices and kernel regression in the quadratic regime
Authors:
Parthe Pandit,
Zhichao Wang,
Yizhe Zhu
Abstract:
Kernel ridge regression (KRR) is a popular class of machine learning models that has become an important tool for understanding deep learning. Much of the focus has been on studying the proportional asymptotic regime, $n \asymp d$, where $n$ is the number of training samples and $d$ is the dimension of the dataset. In this regime, under certain conditions on the data distribution, the kernel rando…
▽ More
Kernel ridge regression (KRR) is a popular class of machine learning models that has become an important tool for understanding deep learning. Much of the focus has been on studying the proportional asymptotic regime, $n \asymp d$, where $n$ is the number of training samples and $d$ is the dimension of the dataset. In this regime, under certain conditions on the data distribution, the kernel random matrix involved in KRR exhibits behavior akin to that of a linear kernel. In this work, we extend the study of kernel regression to the quadratic asymptotic regime, where $n \asymp d^2$. In this regime, we demonstrate that a broad class of inner-product kernels exhibit behavior similar to a quadratic kernel. Specifically, we establish an operator norm approximation bound for the difference between the original kernel random matrix and a quadratic kernel random matrix with additional correction terms compared to the Taylor expansion of the kernel functions. The approximation works for general data distributions under a Gaussian-moment-matching assumption with a covariance structure. This new approximation is utilized to obtain a limiting spectral distribution of the original kernel matrix and characterize the precise asymptotic training and generalization errors for KRR in the quadratic regime when $n/d^2$ converges to a non-zero constant. The generalization errors are obtained for both deterministic and random teacher models. Our proof techniques combine moment methods, Wick's formula, orthogonal polynomials, and resolvent analysis of random matrices with correlated entries.
△ Less
Submitted 2 August, 2024;
originally announced August 2024.
-
Fuzzy Calculus with Noval Approach Using Fuzzy Functions
Authors:
Purnima Pandit,
Payal Singh
Abstract:
This article deals with the complexity involved in fuzzy derivatives when both input and output are from nonempty, convex, and compact fuzzy space. Consider a fuzzy valued mapping, and for fuzzy differentiation of fuzzy valued function, we propose Modified Hukuhara derivative. To evaluate this derivative, we need to take the parametric form of, input and the mapping which is involved in it. Our de…
▽ More
This article deals with the complexity involved in fuzzy derivatives when both input and output are from nonempty, convex, and compact fuzzy space. Consider a fuzzy valued mapping, and for fuzzy differentiation of fuzzy valued function, we propose Modified Hukuhara derivative. To evaluate this derivative, we need to take the parametric form of, input and the mapping which is involved in it. Our definition gives a more realistic explanation of fuzzy derivatives, under this derivative, we also develop fuzzy Taylor series along with its convergence. Lastly, we solve a fully fuzzy differential equation with initial condition using Fuzzy Taylor series.
△ Less
Submitted 22 August, 2023; v1 submitted 6 April, 2023;
originally announced April 2023.
-
High-Dimensional Bernoulli Autoregressive Process with Long-Range Dependence
Authors:
Parthe Pandit,
Mojtaba Sahraee-Ardakan,
Arash A. Amini,
Sundeep Rangan,
Alyson K. Fletcher
Abstract:
We consider the problem of estimating the parameters of a multivariate Bernoulli process with auto-regressive feedback in the high-dimensional setting where the number of samples available is much less than the number of parameters. This problem arises in learning interconnections of networks of dynamical systems with spiking or binary-valued data. We allow the process to depend on its past up to…
▽ More
We consider the problem of estimating the parameters of a multivariate Bernoulli process with auto-regressive feedback in the high-dimensional setting where the number of samples available is much less than the number of parameters. This problem arises in learning interconnections of networks of dynamical systems with spiking or binary-valued data. We allow the process to depend on its past up to a lag $p$, for a general $p \ge 1$, allowing for more realistic modeling in many applications. We propose and analyze an $\ell_1$-regularized maximum likelihood estimator (MLE) under the assumption that the parameter tensor is approximately sparse. Rigorous analysis of such estimators is made challenging by the dependent and non-Gaussian nature of the process as well as the presence of the nonlinearities and multi-level feedback. We derive precise upper bounds on the mean-squared estimation error in terms of the number of samples, dimensions of the process, the lag $p$ and other key statistical properties of the model. The ideas presented can be used in the high-dimensional analysis of regularized $M$-estimators for other sparse nonlinear and non-Gaussian processes with long-range dependence.
△ Less
Submitted 19 March, 2019;
originally announced March 2019.
-
Iterated logarithms and gradient flows
Authors:
Fabian Haiden,
Ludmil Katzarkov,
Maxim Kontsevich,
Pranav Pandit
Abstract:
We consider applications of the theory of balanced weight filtrations and iterated logarithms, initiated in arXiv:1706.01073, to PDEs. The main result is a complete description of the asymptotics of the Yang--Mills flow on the space of metrics on a holomorphic bundle over a Riemann surface. A key ingredient in the argument is a monotonicity property of the flow which holds in arbitrary dimension.…
▽ More
We consider applications of the theory of balanced weight filtrations and iterated logarithms, initiated in arXiv:1706.01073, to PDEs. The main result is a complete description of the asymptotics of the Yang--Mills flow on the space of metrics on a holomorphic bundle over a Riemann surface. A key ingredient in the argument is a monotonicity property of the flow which holds in arbitrary dimension. The A-side analog is a modified curve shortening flow for which we provide a heuristic calculation in support of a detailed conjectural picture.
△ Less
Submitted 12 February, 2018;
originally announced February 2018.
-
Semistability, modular lattices, and iterated logarithms
Authors:
Fabian Haiden,
Ludmil Katzarkov,
Maxim Kontsevich,
Pranav Pandit
Abstract:
We provide a complete description of the asymptotics of the gradient flow on the space of metrics on any semistable quiver representation. This involves a recursive construction of approximate solutions and the appearance of iterated logarithms and a limiting filtration of the representation. The filtration turns out to have an algebraic definition which makes sense in any finite length modular la…
▽ More
We provide a complete description of the asymptotics of the gradient flow on the space of metrics on any semistable quiver representation. This involves a recursive construction of approximate solutions and the appearance of iterated logarithms and a limiting filtration of the representation. The filtration turns out to have an algebraic definition which makes sense in any finite length modular lattice. This is part of a larger project by the authors to study iterated logarithms in the asymptotics of gradient flows, both in finite and infinite dimensional settings.
△ Less
Submitted 10 September, 2020; v1 submitted 4 June, 2017;
originally announced June 2017.
-
Generators in formal deformations of categories
Authors:
Anthony Blanc,
Ludmil Katzarkov,
Pranav Pandit
Abstract:
In this paper we use the theory of formal moduli problems developed by Lurie in order to study the space of formal deformations of a $k$-linear $\infty$-category for a field $k$. Our main result states that if $\mathcal{C}$ is a $k$-linear $\infty$-category which has a compact generator whose groups of self extensions vanish for sufficiently high positive degrees, then every formal deformation of…
▽ More
In this paper we use the theory of formal moduli problems developed by Lurie in order to study the space of formal deformations of a $k$-linear $\infty$-category for a field $k$. Our main result states that if $\mathcal{C}$ is a $k$-linear $\infty$-category which has a compact generator whose groups of self extensions vanish for sufficiently high positive degrees, then every formal deformation of $\mathcal{C}$ has zero curvature and moreover admits a compact generator.
△ Less
Submitted 1 May, 2017;
originally announced May 2017.
-
Calabi-Yau Structures, Spherical Functors, and Shifted Symplectic Structures
Authors:
Ludmil Katzarkov,
Pranav Pandit,
Theodore Spaide
Abstract:
A categorical formalism is introduced for studying various features of the symplectic geometry of Lefschetz fibrations and the algebraic geometry of Tyurin degenerations. This approach is informed by homological mirror symmetry, derived noncommutative geometry, and the theory of Fukaya categories with coefficients in a perverse Schober. The main technical results include (i) a comparison between t…
▽ More
A categorical formalism is introduced for studying various features of the symplectic geometry of Lefschetz fibrations and the algebraic geometry of Tyurin degenerations. This approach is informed by homological mirror symmetry, derived noncommutative geometry, and the theory of Fukaya categories with coefficients in a perverse Schober. The main technical results include (i) a comparison between the notion of relative Calabi-Yau structures and a certain refinement of the notion of a spherical functor, (ii) a local-to-global gluing principle for constructing Calabi-Yau structures, and (iii) the construction of shifted symplectic structures and Lagrangian structures on certain derived moduli spaces of branes. Potential applications to a theory of derived hyperkähler geometry are sketched.
△ Less
Submitted 3 September, 2017; v1 submitted 26 January, 2017;
originally announced January 2017.
-
Reduction for $SL(3)$ pre-buildings
Authors:
Ludmil Katzarkov,
Pranav Pandit,
Carlos Simpson
Abstract:
Given an $SL(3)$ spectral curve over a simply connected Riemann surface, we describe in detail the reduction steps necessary to construct the core of a pre-building with versal harmonic map whose differential is given by the spectral curve.
Given an $SL(3)$ spectral curve over a simply connected Riemann surface, we describe in detail the reduction steps necessary to construct the core of a pre-building with versal harmonic map whose differential is given by the spectral curve.
△ Less
Submitted 25 November, 2016;
originally announced November 2016.
-
Refinement of the Equilibrium of Public Goods Games over Networks: Efficiency and Effort of Specialized Equilibria
Authors:
Parthe Pandit,
Ankur A. Kulkarni
Abstract:
Recently Bramoulle and Kranton presented a model for the provision of public goods over a network and showed the existence of a class of Nash equilibria called specialized equilibria wherein some agents exert maximum effort while other agents free ride. We examine the efficiency, effort and cost of specialized equilibria in comparison to other equilibria. Our main results show that the welfare of…
▽ More
Recently Bramoulle and Kranton presented a model for the provision of public goods over a network and showed the existence of a class of Nash equilibria called specialized equilibria wherein some agents exert maximum effort while other agents free ride. We examine the efficiency, effort and cost of specialized equilibria in comparison to other equilibria. Our main results show that the welfare of a particular specialized equilibrium approaches the maximum welfare amongst all equilibria as the concavity of the benefit function tends to unity. For forest networks a similar result also holds as the concavity approaches zero. Moreover, without any such concavity conditions, there exists for any network a specialized equilibrium that requires the maximum weighted effort amongst all equilibria. When the network is a forest, a specialized equilibrium also incurs the minimum total cost amongst all equilibria. For well-covered forest networks we show that all welfare maximizing equilibria are specialized and all equilibria incur the same total cost. Thus we argue that specialized equilibria may be considered as a refinement of the equilibrium of the public goods game. We show several results on the structure and efficiency of equilibria that highlight the role of dependants in the network.
△ Less
Submitted 23 January, 2022; v1 submitted 7 July, 2016;
originally announced July 2016.
-
A linear complementarity based characterization of the weighted independence number and the independent domination number in graphs
Authors:
Parthe Pandit,
Ankur A. Kulkarni
Abstract:
The linear complementarity problem is a continuous optimization problem that generalizes convex quadratic programming, Nash equilibria of bimatrix games and several such problems. This paper presents a continuous optimization formulation for the weighted independence number of a graph by characterizing it as the maximum weighted $\ell_1$ norm over the solution set of a linear complementarity probl…
▽ More
The linear complementarity problem is a continuous optimization problem that generalizes convex quadratic programming, Nash equilibria of bimatrix games and several such problems. This paper presents a continuous optimization formulation for the weighted independence number of a graph by characterizing it as the maximum weighted $\ell_1$ norm over the solution set of a linear complementarity problem (LCP). The minimum $\ell_1$ norm of solutions of this LCP is a lower bound on the independent domination number of the graph. Unlike the case of the maximum $\ell_1$ norm, this lower bound is in general weak, but we show it to be tight if the graph is a forest. Using methods from the theory of LCPs, we obtain a few graph theoretic results. In particular, we provide a stronger variant of the Lovász theta of a graph. We then provide sufficient conditions for a graph to be well-covered, i.e., for all maximal independent sets to also be maximum. This condition is also shown to be necessary for well-coveredness if the graph is a forest. Finally, the reduction of the maximum independent set problem to a linear program with (linear) complementarity constraints (LPCC) shows that LPCCs are hard to approximate.
△ Less
Submitted 16 March, 2016;
originally announced March 2016.
-
Constructing Buildings and Harmonic Maps
Authors:
Ludmil Katzarkov,
Alexander Noll,
Pranav Pandit,
Carlos Simpson
Abstract:
In a continuation of our previous work, we outline a theory which should lead to the construction of a universal pre-building and versal building with a $φ$-harmonic map from a Riemann surface, in the case of two-dimensional buildings for the group $SL_3$. This will provide a generalization of the space of leaves of the foliation defined by a quadratic differential in the classical theory for…
▽ More
In a continuation of our previous work, we outline a theory which should lead to the construction of a universal pre-building and versal building with a $φ$-harmonic map from a Riemann surface, in the case of two-dimensional buildings for the group $SL_3$. This will provide a generalization of the space of leaves of the foliation defined by a quadratic differential in the classical theory for $SL_2$. Our conjectural construction would determine the exponents for $SL_3$ WKB problems, and it can be put into practice on examples.
△ Less
Submitted 3 March, 2015;
originally announced March 2015.
-
Harmonic Maps to Buildings and Singular Perturbation Theory
Authors:
Ludmil Katzarkov,
Alexander Noll,
Pranav Pandit,
Carlos Simpson
Abstract:
The notion of a universal building associated with a point in the Hitchin base is introduced. This is a building equipped with a harmonic map from a Riemann surface that is initial among harmonic maps which induce the given cameral cover of the Riemann surface. In the rank one case, the universal building is the leaf space of the quadratic differential defining the point in the Hitchin base.
The…
▽ More
The notion of a universal building associated with a point in the Hitchin base is introduced. This is a building equipped with a harmonic map from a Riemann surface that is initial among harmonic maps which induce the given cameral cover of the Riemann surface. In the rank one case, the universal building is the leaf space of the quadratic differential defining the point in the Hitchin base.
The main conjectures of this paper are: (1) the universal building always exists; (2) the harmonic map to the universal building controls the asymptotics of the Riemann-Hilbert correspondence and the non-abelian Hodge correspondence; (3) the singularities of the universal building give rise to Spectral Networks; and (4) the universal building encodes the data of a 3d Calabi-Yau category whose space of stability conditions has a connected component that contains the Hitchin base.
The main theorem establishes the existence of the universal building, conjecture (3), as well as the Riemann-Hilbert part of conjecture (2), in the case of the rank two example introduced in the seminal work of Berk-Nevins-Roberts on higher order Stokes phenomena. It is also shown that the asymptotics of the Riemann-Hilbert correspondence is always controlled by a harmonic map to a certain building, which is constructed as the asymptotic cone of a symmetric space.
△ Less
Submitted 27 November, 2013;
originally announced November 2013.
