Showing 1–47 of 47 results for author: Perlmutter, M

InfoGain Wavelets: Furthering the Design of Diffusion Wavelets for Graph-Structured Data

Authors: David R. Johnson, Smita Krishnaswamy, Michael Perlmutter

Abstract: Diffusion wavelets extract information from graph signals at different scales of resolution by utilizing graph diffusion operators raised to various powers, known as diffusion scales. Traditionally, the diffusion scales are chosen to be dyadic integers, $\mathbf{2^j}$. Here, we propose a novel, unsupervised method for selecting the diffusion scales based on ideas from information theory. We then s… ▽ More Diffusion wavelets extract information from graph signals at different scales of resolution by utilizing graph diffusion operators raised to various powers, known as diffusion scales. Traditionally, the diffusion scales are chosen to be dyadic integers, $\mathbf{2^j}$. Here, we propose a novel, unsupervised method for selecting the diffusion scales based on ideas from information theory. We then show that our method can be incorporated into wavelet-based GNNs via graph classification experiments. △ Less

Submitted 8 April, 2025; originally announced April 2025.

Comments: This work was accepted to be presented at the Graph Signal Processing Workshop 2025

HiPoNet: A Multi-View Simplicial Complex Network for High Dimensional Point-Cloud and Single-Cell Data

Authors: Siddharth Viswanath, Hiren Madhu, Dhananjay Bhaskar, Jake Kovalic, David R Johnson, Christopher Tape, Ian Adelstein, Rex Ying, Michael Perlmutter, Smita Krishnaswamy

Abstract: In this paper, we propose HiPoNet, an end-to-end differentiable neural network for regression, classification, and representation learning on high-dimensional point clouds. Our work is motivated by single-cell data which can have very high-dimensionality --exceeding the capabilities of existing methods for point clouds which are mostly tailored for 3D data. Moreover, modern single-cell and spatial… ▽ More In this paper, we propose HiPoNet, an end-to-end differentiable neural network for regression, classification, and representation learning on high-dimensional point clouds. Our work is motivated by single-cell data which can have very high-dimensionality --exceeding the capabilities of existing methods for point clouds which are mostly tailored for 3D data. Moreover, modern single-cell and spatial experiments now yield entire cohorts of datasets (i.e., one data set for every patient), necessitating models that can process large, high-dimensional point-clouds at scale. Most current approaches build a single nearest-neighbor graph, discarding important geometric and topological information. In contrast, HiPoNet models the point-cloud as a set of higher-order simplicial complexes, with each particular complex being created using a reweighting of features. This method thus generates multiple constructs corresponding to different views of high-dimensional data, which in biology offers the possibility of disentangling distinct cellular processes. It then employs simplicial wavelet transforms to extract multiscale features, capturing both local and global topology from each view. We show that geometric and topological information is preserved in this framework both theoretically and empirically. We showcase the utility of HiPoNet on point-cloud level tasks, involving classification and regression of entire point-clouds in data cohorts. Experimentally, we find that HiPoNet outperforms other point-cloud and graph-based models on single-cell data. We also apply HiPoNet to spatial transcriptomics datasets using spatial coordinates as one of the views. Overall, HiPoNet offers a robust and scalable solution for high-dimensional data analysis. △ Less

Submitted 26 May, 2025; v1 submitted 11 February, 2025; originally announced February 2025.

ProtSCAPE: Mapping the landscape of protein conformations in molecular dynamics

Authors: Siddharth Viswanath, Dhananjay Bhaskar, David R. Johnson, Joao Felipe Rocha, Egbert Castro, Jackson D. Grady, Alex T. Grigas, Michael A. Perlmutter, Corey S. O'Hern, Smita Krishnaswamy

Abstract: Understanding the dynamic nature of protein structures is essential for comprehending their biological functions. While significant progress has been made in predicting static folded structures, modeling protein motions on microsecond to millisecond scales remains challenging. To address these challenges, we introduce a novel deep learning architecture, Protein Transformer with Scattering, Attenti… ▽ More Understanding the dynamic nature of protein structures is essential for comprehending their biological functions. While significant progress has been made in predicting static folded structures, modeling protein motions on microsecond to millisecond scales remains challenging. To address these challenges, we introduce a novel deep learning architecture, Protein Transformer with Scattering, Attention, and Positional Embedding (ProtSCAPE), which leverages the geometric scattering transform alongside transformer-based attention mechanisms to capture protein dynamics from molecular dynamics (MD) simulations. ProtSCAPE utilizes the multi-scale nature of the geometric scattering transform to extract features from protein structures conceptualized as graphs and integrates these features with dual attention structures that focus on residues and amino acid signals, generating latent representations of protein trajectories. Furthermore, ProtSCAPE incorporates a regression head to enforce temporally coherent latent representations. △ Less

Submitted 26 October, 2024; originally announced October 2024.

Comments: Accepted as a short paper at the 5th Molecular Machine Learning Conference (MoML 2024)

Convergence of Manifold Filter-Combine Networks

Authors: David R. Johnson, Joyce Chew, Siddharth Viswanath, Edward De Brouwer, Deanna Needell, Smita Krishnaswamy, Michael Perlmutter

Abstract: In order to better understand manifold neural networks (MNNs), we introduce Manifold Filter-Combine Networks (MFCNs). The filter-combine framework parallels the popular aggregate-combine paradigm for graph neural networks (GNNs) and naturally suggests many interesting families of MNNs which can be interpreted as the manifold analog of various popular GNNs. We then propose a method for implementing… ▽ More In order to better understand manifold neural networks (MNNs), we introduce Manifold Filter-Combine Networks (MFCNs). The filter-combine framework parallels the popular aggregate-combine paradigm for graph neural networks (GNNs) and naturally suggests many interesting families of MNNs which can be interpreted as the manifold analog of various popular GNNs. We then propose a method for implementing MFCNs on high-dimensional point clouds that relies on approximating the manifold by a sparse graph. We prove that our method is consistent in the sense that it converges to a continuum limit as the number of data points tends to infinity. △ Less

Submitted 18 October, 2024; originally announced October 2024.

Comments: Accepted to NeurIPS Workshop on Symmetry and Geometry in Neural Representations (Extended Abstract Track)

Hyperedge Representations with Hypergraph Wavelets: Applications to Spatial Transcriptomics

Authors: Xingzhi Sun, Charles Xu, João F. Rocha, Chen Liu, Benjamin Hollander-Bodie, Laney Goldman, Marcello DiStasio, Michael Perlmutter, Smita Krishnaswamy

Abstract: In many data-driven applications, higher-order relationships among multiple objects are essential in capturing complex interactions. Hypergraphs, which generalize graphs by allowing edges to connect any number of nodes, provide a flexible and powerful framework for modeling such higher-order relationships. In this work, we introduce hypergraph diffusion wavelets and describe their favorable spectr… ▽ More In many data-driven applications, higher-order relationships among multiple objects are essential in capturing complex interactions. Hypergraphs, which generalize graphs by allowing edges to connect any number of nodes, provide a flexible and powerful framework for modeling such higher-order relationships. In this work, we introduce hypergraph diffusion wavelets and describe their favorable spectral and spatial properties. We demonstrate their utility for biomedical discovery in spatially resolved transcriptomics by applying the method to represent disease-relevant cellular niches for Alzheimer's disease. △ Less

Submitted 14 September, 2024; originally announced September 2024.

Towards a General Recipe for Combinatorial Optimization with Multi-Filter GNNs

Authors: Frederik Wenkel, Semih Cantürk, Stefan Horoi, Michael Perlmutter, Guy Wolf

Abstract: Graph neural networks (GNNs) have achieved great success for a variety of tasks such as node classification, graph classification, and link prediction. However, the use of GNNs (and machine learning more generally) to solve combinatorial optimization (CO) problems is much less explored. Here, we introduce GCON, a novel GNN architecture that leverages a complex filter bank and localized attention m… ▽ More Graph neural networks (GNNs) have achieved great success for a variety of tasks such as node classification, graph classification, and link prediction. However, the use of GNNs (and machine learning more generally) to solve combinatorial optimization (CO) problems is much less explored. Here, we introduce GCON, a novel GNN architecture that leverages a complex filter bank and localized attention mechanisms to solve CO problems on graphs. We show how our method differentiates itself from prior GNN-based CO solvers and how it can be effectively applied to the maximum cut, minimum dominating set, and maximum clique problems in a unsupervised learning setting. GCON is competitive across all tasks and consistently outperforms other specialized GNN-based approaches, and is on par with the powerful Gurobi solver on the max-cut problem. We provide an open-source implementation of our work at https://github.com/WenkelF/copt. △ Less

Submitted 24 November, 2024; v1 submitted 30 May, 2024; originally announced May 2024.

Comments: In Proceedings of the Third Learning on Graphs Conference (LoG 2024, Oral); 20 pages, 2 figures

MSC Class: 68T07 (Primary) 68T20; 90C35; 05C62 (Secondary) ACM Class: F.2.2; I.2.6

Bayesian Formulations for Graph Spectral Denoising

Authors: Sam Leone, Xingzhi Sun, Michael Perlmutter, Smita Krishnaswamy

Abstract: Here we consider the problem of denoising features associated to complex data, modeled as signals on a graph, via a smoothness prior. This is motivated in part by settings such as single-cell RNA where the data is very high-dimensional, but its structure can be captured via an affinity graph. This allows us to utilize ideas from graph signal processing. In particular, we present algorithms for the… ▽ More Here we consider the problem of denoising features associated to complex data, modeled as signals on a graph, via a smoothness prior. This is motivated in part by settings such as single-cell RNA where the data is very high-dimensional, but its structure can be captured via an affinity graph. This allows us to utilize ideas from graph signal processing. In particular, we present algorithms for the cases where the signal is perturbed by Gaussian noise, dropout, and uniformly distributed noise. The signals are assumed to follow a prior distribution defined in the frequency domain which favors signals which are smooth across the edges of the graph. By pairing this prior distribution with our three models of noise generation, we propose Maximum A Posteriori (M.A.P.) estimates of the true signal in the presence of noisy data and provide algorithms for computing the M.A.P. Finally, we demonstrate the algorithms' ability to effectively restore signals from white noise on image data and from severe dropout in single-cell RNA sequence data. △ Less

Submitted 8 December, 2023; v1 submitted 27 November, 2023; originally announced November 2023.

BLIS-Net: Classifying and Analyzing Signals on Graphs

Authors: Charles Xu, Laney Goldman, Valentina Guo, Benjamin Hollander-Bodie, Maedee Trank-Greene, Ian Adelstein, Edward De Brouwer, Rex Ying, Smita Krishnaswamy, Michael Perlmutter

Abstract: Graph neural networks (GNNs) have emerged as a powerful tool for tasks such as node classification and graph classification. However, much less work has been done on signal classification, where the data consists of many functions (referred to as signals) defined on the vertices of a single graph. These tasks require networks designed differently from those designed for traditional GNN tasks. Inde… ▽ More Graph neural networks (GNNs) have emerged as a powerful tool for tasks such as node classification and graph classification. However, much less work has been done on signal classification, where the data consists of many functions (referred to as signals) defined on the vertices of a single graph. These tasks require networks designed differently from those designed for traditional GNN tasks. Indeed, traditional GNNs rely on localized low-pass filters, and signals of interest may have intricate multi-frequency behavior and exhibit long range interactions. This motivates us to introduce the BLIS-Net (Bi-Lipschitz Scattering Net), a novel GNN that builds on the previously introduced geometric scattering transform. Our network is able to capture both local and global signal structure and is able to capture both low-frequency and high-frequency information. We make several crucial changes to the original geometric scattering architecture which we prove increase the ability of our network to capture information about the input signal and show that BLIS-Net achieves superior performance on both synthetic and real-world data sets based on traffic flow and fMRI data. △ Less

Submitted 26 October, 2023; originally announced October 2023.

Journal ref: Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, PMLR 238:4537-4545, 2024

DYMAG: Rethinking Message Passing Using Dynamical-systems-based Waveforms

Authors: Dhananjay Bhaskar, Xingzhi Sun, Yanlei Zhang, Charles Xu, Arman Afrasiyabi, Siddharth Viswanath, Oluwadamilola Fasina, Maximilian Nickel, Guy Wolf, Michael Perlmutter, Smita Krishnaswamy

Abstract: We present DYMAG, a graph neural network based on a novel form of message aggregation. Standard message-passing neural networks, which often aggregate local neighbors via mean-aggregation, can be regarded as convolving with a simple rectangular waveform which is non-zero only on 1-hop neighbors of every vertex. Here, we go beyond such local averaging. We will convolve the node features with more s… ▽ More We present DYMAG, a graph neural network based on a novel form of message aggregation. Standard message-passing neural networks, which often aggregate local neighbors via mean-aggregation, can be regarded as convolving with a simple rectangular waveform which is non-zero only on 1-hop neighbors of every vertex. Here, we go beyond such local averaging. We will convolve the node features with more sophisticated waveforms generated using dynamics such as the heat equation, wave equation, and the Sprott model (an example of chaotic dynamics). Furthermore, we use snapshots of these dynamics at different time points to create waveforms at many effective scales. Theoretically, we show that these dynamic waveforms can capture salient information about the graph including connected components, connectivity, and cycle structures even with no features. Empirically, we test DYMAG on both real and synthetic benchmarks to establish that DYMAG outperforms baseline models on recovery of graph persistence, generating parameters of random graphs, as well as property prediction for proteins, molecules and materials. Our code is available at https://github.com/KrishnaswamyLab/DYMAG. △ Less

Submitted 26 May, 2025; v1 submitted 18 September, 2023; originally announced September 2023.

Directed Scattering for Knowledge Graph-based Cellular Signaling Analysis

Authors: Aarthi Venkat, Joyce Chew, Ferran Cardoso Rodriguez, Christopher J. Tape, Michael Perlmutter, Smita Krishnaswamy

Abstract: Directed graphs are a natural model for many phenomena, in particular scientific knowledge graphs such as molecular interaction or chemical reaction networks that define cellular signaling relationships. In these situations, source nodes typically have distinct biophysical properties from sinks. Due to their ordered and unidirectional relationships, many such networks also have hierarchical and mu… ▽ More Directed graphs are a natural model for many phenomena, in particular scientific knowledge graphs such as molecular interaction or chemical reaction networks that define cellular signaling relationships. In these situations, source nodes typically have distinct biophysical properties from sinks. Due to their ordered and unidirectional relationships, many such networks also have hierarchical and multiscale structure. However, the majority of methods performing node- and edge-level tasks in machine learning do not take these properties into account, and thus have not been leveraged effectively for scientific tasks such as cellular signaling network inference. We propose a new framework called Directed Scattering Autoencoder (DSAE) which uses a directed version of a geometric scattering transform, combined with the non-linear dimensionality reduction properties of an autoencoder and the geometric properties of the hyperbolic space to learn latent hierarchies. We show this method outperforms numerous others on tasks such as embedding directed graphs and learning cellular signaling networks. △ Less

Submitted 14 September, 2023; originally announced September 2023.

Comments: 5 pages, 3 figures

A Flow Artist for High-Dimensional Cellular Data

Authors: Kincaid MacDonald, Dhananjay Bhaskar, Guy Thampakkul, Nhi Nguyen, Joia Zhang, Michael Perlmutter, Ian Adelstein, Smita Krishnaswamy

Abstract: We consider the problem of embedding point cloud data sampled from an underlying manifold with an associated flow or velocity. Such data arises in many contexts where static snapshots of dynamic entities are measured, including in high-throughput biology such as single-cell transcriptomics. Existing embedding techniques either do not utilize velocity information or embed the coordinates and veloci… ▽ More We consider the problem of embedding point cloud data sampled from an underlying manifold with an associated flow or velocity. Such data arises in many contexts where static snapshots of dynamic entities are measured, including in high-throughput biology such as single-cell transcriptomics. Existing embedding techniques either do not utilize velocity information or embed the coordinates and velocities independently, i.e., they either impose velocities on top of an existing point embedding or embed points within a prescribed vector field. Here we present FlowArtist, a neural network that embeds points while jointly learning a vector field around the points. The combination allows FlowArtist to better separate and visualize velocity-informed structures. Our results, on toy datasets and single-cell RNA velocity data, illustrate the value of utilizing coordinate and velocity information in tandem for embedding and visualizing high-dimensional data. △ Less

Submitted 31 July, 2023; originally announced August 2023.

Comments: Accepted for publication in 2023 IEEE 33rd International Workshop on Machine Learning for Signal Processing (MLSP)

Manifold Filter-Combine Networks

Authors: David R. Johnson, Joyce A. Chew, Edward De Brouwer, Smita Krishnaswamy, Deanna Needell, Michael Perlmutter

Abstract: In order to better understand manifold neural networks (MNNs), we introduce Manifold Filter-Combine Networks (MFCNs). Our filter-combine framework parallels the popular aggregate-combine paradigm for graph neural networks (GNNs) and naturally suggests many interesting families of MNNs which can be interpreted as manifold analogues of various popular GNNs. We propose a method for implementing MFCNs… ▽ More In order to better understand manifold neural networks (MNNs), we introduce Manifold Filter-Combine Networks (MFCNs). Our filter-combine framework parallels the popular aggregate-combine paradigm for graph neural networks (GNNs) and naturally suggests many interesting families of MNNs which can be interpreted as manifold analogues of various popular GNNs. We propose a method for implementing MFCNs on high-dimensional point clouds that relies on approximating an underlying manifold by a sparse graph. We then prove that our method is consistent in the sense that it converges to a continuum limit as the number of data points tends to infinity, and we numerically demonstrate its effectiveness on real-world and synthetic data sets. △ Less

Submitted 7 January, 2025; v1 submitted 8 July, 2023; originally announced July 2023.

A Convergence Rate for Manifold Neural Networks

Authors: Joyce Chew, Deanna Needell, Michael Perlmutter

Abstract: High-dimensional data arises in numerous applications, and the rapidly developing field of geometric deep learning seeks to develop neural network architectures to analyze such data in non-Euclidean domains, such as graphs and manifolds. Recent work by Z. Wang, L. Ruiz, and A. Ribeiro has introduced a method for constructing manifold neural networks using the spectral decomposition of the Laplace… ▽ More High-dimensional data arises in numerous applications, and the rapidly developing field of geometric deep learning seeks to develop neural network architectures to analyze such data in non-Euclidean domains, such as graphs and manifolds. Recent work by Z. Wang, L. Ruiz, and A. Ribeiro has introduced a method for constructing manifold neural networks using the spectral decomposition of the Laplace Beltrami operator. Moreover, in this work, the authors provide a numerical scheme for implementing such neural networks when the manifold is unknown and one only has access to finitely many sample points. The authors show that this scheme, which relies upon building a data-driven graph, converges to the continuum limit as the number of sample points tends to infinity. Here, we build upon this result by establishing a rate of convergence that depends on the intrinsic dimension of the manifold but is independent of the ambient dimension. We also discuss how the rate of convergence depends on the depth of the network and the number of filters used in each layer. △ Less

Submitted 20 July, 2023; v1 submitted 23 December, 2022; originally announced December 2022.

Multi-scale Hybridized Topic Modeling: A Pipeline for Analyzing Unstructured Text Datasets via Topic Modeling

Authors: Keyi Cheng, Stefan Inzer, Adrian Leung, Xiaoxian Shen, Michael Perlmutter, Michael Lindstrom, Joyce Chew, Todd Presner, Deanna Needell

Abstract: We propose a multi-scale hybridized topic modeling method to find hidden topics from transcribed interviews more accurately and efficiently than traditional topic modeling methods. Our multi-scale hybridized topic modeling method (MSHTM) approaches data at different scales and performs topic modeling in a hierarchical way utilizing first a classical method, Nonnegative Matrix Factorization, and th… ▽ More We propose a multi-scale hybridized topic modeling method to find hidden topics from transcribed interviews more accurately and efficiently than traditional topic modeling methods. Our multi-scale hybridized topic modeling method (MSHTM) approaches data at different scales and performs topic modeling in a hierarchical way utilizing first a classical method, Nonnegative Matrix Factorization, and then a transformer-based method, BERTopic. It harnesses the strengths of both NMF and BERTopic. Our method can help researchers and the public better extract and interpret the interview information. Additionally, it provides insights for new indexing systems based on the topic level. We then deploy our method on real-world interview transcripts and find promising results. △ Less

Submitted 24 November, 2022; originally announced November 2022.

Geometric Scattering on Measure Spaces

Authors: Joyce Chew, Matthew Hirn, Smita Krishnaswamy, Deanna Needell, Michael Perlmutter, Holly Steach, Siddharth Viswanath, Hau-Tieng Wu

Abstract: The scattering transform is a multilayered, wavelet-based transform initially introduced as a model of convolutional neural networks (CNNs) that has played a foundational role in our understanding of these networks' stability and invariance properties. Subsequently, there has been widespread interest in extending the success of CNNs to data sets with non-Euclidean structure, such as graphs and man… ▽ More The scattering transform is a multilayered, wavelet-based transform initially introduced as a model of convolutional neural networks (CNNs) that has played a foundational role in our understanding of these networks' stability and invariance properties. Subsequently, there has been widespread interest in extending the success of CNNs to data sets with non-Euclidean structure, such as graphs and manifolds, leading to the emerging field of geometric deep learning. In order to improve our understanding of the architectures used in this new field, several papers have proposed generalizations of the scattering transform for non-Euclidean data structures such as undirected graphs and compact Riemannian manifolds without boundary. In this paper, we introduce a general, unified model for geometric scattering on measure spaces. Our proposed framework includes previous work on geometric scattering as special cases but also applies to more general settings such as directed graphs, signed graphs, and manifolds with boundary. We propose a new criterion that identifies to which groups a useful representation should be invariant and show that this criterion is sufficient to guarantee that the scattering transform has desirable stability and invariance properties. Additionally, we consider finite measure spaces that are obtained from randomly sampling an unknown manifold. We propose two methods for constructing a data-driven graph on which the associated graph scattering transform approximates the scattering transform on the underlying manifold. Moreover, we use a diffusion-maps based approach to prove quantitative estimates on the rate of convergence of one of these approximations as the number of sample points tends to infinity. Lastly, we showcase the utility of our method on spherical images, directed graphs, and on high-dimensional single-cell data. △ Less

Submitted 13 January, 2025; v1 submitted 17 August, 2022; originally announced August 2022.

MSC Class: 68T07

Learnable Filters for Geometric Scattering Modules

Authors: Alexander Tong, Frederik Wenkel, Dhananjay Bhaskar, Kincaid Macdonald, Jackson Grady, Michael Perlmutter, Smita Krishnaswamy, Guy Wolf

Abstract: We propose a new graph neural network (GNN) module, based on relaxations of recently proposed geometric scattering transforms, which consist of a cascade of graph wavelet filters. Our learnable geometric scattering (LEGS) module enables adaptive tuning of the wavelets to encourage band-pass features to emerge in learned representations. The incorporation of our LEGS-module in GNNs enables the lear… ▽ More We propose a new graph neural network (GNN) module, based on relaxations of recently proposed geometric scattering transforms, which consist of a cascade of graph wavelet filters. Our learnable geometric scattering (LEGS) module enables adaptive tuning of the wavelets to encourage band-pass features to emerge in learned representations. The incorporation of our LEGS-module in GNNs enables the learning of longer-range graph relations compared to many popular GNNs, which often rely on encoding graph structure via smoothness or similarity between neighbors. Further, its wavelet priors result in simplified architectures with significantly fewer learned parameters compared to competing GNNs. We demonstrate the predictive performance of LEGS-based networks on graph classification benchmarks, as well as the descriptive quality of their learned features in biochemical graph data exploration tasks. Our results show that LEGS-based networks match or outperforms popular GNNs, as well as the original geometric scattering construction, on many datasets, in particular in biochemical domains, while retaining certain mathematical properties of handcrafted (non-learned) geometric scattering. △ Less

Submitted 15 August, 2022; originally announced August 2022.

Comments: 14 pages, 3 figures, 10 tables. arXiv admin note: substantial text overlap with arXiv:2010.02415

The Manifold Scattering Transform for High-Dimensional Point Cloud Data

Authors: Joyce Chew, Holly R. Steach, Siddharth Viswanath, Hau-Tieng Wu, Matthew Hirn, Deanna Needell, Smita Krishnaswamy, Michael Perlmutter

Abstract: The manifold scattering transform is a deep feature extractor for data defined on a Riemannian manifold. It is one of the first examples of extending convolutional neural network-like operators to general manifolds. The initial work on this model focused primarily on its theoretical stability and invariance properties but did not provide methods for its numerical implementation except in the case… ▽ More The manifold scattering transform is a deep feature extractor for data defined on a Riemannian manifold. It is one of the first examples of extending convolutional neural network-like operators to general manifolds. The initial work on this model focused primarily on its theoretical stability and invariance properties but did not provide methods for its numerical implementation except in the case of two-dimensional surfaces with predefined meshes. In this work, we present practical schemes, based on the theory of diffusion maps, for implementing the manifold scattering transform to datasets arising in naturalistic systems, such as single cell genetics, where the data is a high-dimensional point cloud modeled as lying on a low-dimensional manifold. We show that our methods are effective for signal classification and manifold classification tasks. △ Less

Submitted 21 January, 2024; v1 submitted 20 June, 2022; originally announced June 2022.

Comments: Accepted for publication in the TAG in DS Workshop at ICML. For subsequent theoretical guarantees, please see Section 6 of arXiv:2208.08561

MSC Class: 68T07 ACM Class: I.2.6

Taxonomy of Benchmarks in Graph Representation Learning

Authors: Renming Liu, Semih Cantürk, Frederik Wenkel, Sarah McGuire, Xinyi Wang, Anna Little, Leslie O'Bray, Michael Perlmutter, Bastian Rieck, Matthew Hirn, Guy Wolf, Ladislav Rampášek

Abstract: Graph Neural Networks (GNNs) extend the success of neural networks to graph-structured data by accounting for their intrinsic geometry. While extensive research has been done on developing GNN models with superior performance according to a collection of graph representation learning benchmarks, it is currently not well understood what aspects of a given model are probed by them. For example, to w… ▽ More Graph Neural Networks (GNNs) extend the success of neural networks to graph-structured data by accounting for their intrinsic geometry. While extensive research has been done on developing GNN models with superior performance according to a collection of graph representation learning benchmarks, it is currently not well understood what aspects of a given model are probed by them. For example, to what extent do they test the ability of a model to leverage graph structure vs. node features? Here, we develop a principled approach to taxonomize benchmarking datasets according to a $\textit{sensitivity profile}$ that is based on how much GNN performance changes due to a collection of graph perturbations. Our data-driven analysis provides a deeper understanding of which benchmarking data characteristics are leveraged by GNNs. Consequently, our taxonomy can aid in selection and development of adequate graph benchmarks, and better informed evaluation of future GNN methods. Finally, our approach and implementation in $\texttt{GTaxoGym}$ package are extendable to multiple graph prediction task types and future datasets. △ Less

Submitted 30 November, 2022; v1 submitted 15 June, 2022; originally announced June 2022.

Comments: In Proceedings of the First Learning on Graphs Conference (LoG 2022)

Can Hybrid Geometric Scattering Networks Help Solve the Maximum Clique Problem?

Authors: Yimeng Min, Frederik Wenkel, Michael Perlmutter, Guy Wolf

Abstract: We propose a geometric scattering-based graph neural network (GNN) for approximating solutions of the NP-hard maximum clique (MC) problem. We construct a loss function with two terms, one which encourages the network to find highly connected nodes and the other which acts as a surrogate for the constraint that the nodes form a clique. We then use this loss to train an efficient GNN architecture th… ▽ More We propose a geometric scattering-based graph neural network (GNN) for approximating solutions of the NP-hard maximum clique (MC) problem. We construct a loss function with two terms, one which encourages the network to find highly connected nodes and the other which acts as a surrogate for the constraint that the nodes form a clique. We then use this loss to train an efficient GNN architecture that outputs a vector representing the probability for each node to be part of the MC and apply a rule-based decoder to make our final prediction. The incorporation of the scattering transform alleviates the so-called oversmoothing problem that is often encountered in GNNs and would degrade the performance of our proposed setup. Our empirical results demonstrate that our method outperforms representative GNN baselines in terms of solution accuracy and inference speed as well as conventional solvers like Gurobi with limited time budgets. Furthermore, our scattering model is very parameter efficient with only $\sim$ 0.1\% of the number of parameters compared to previous GNN baseline models. △ Less

Submitted 28 November, 2022; v1 submitted 3 June, 2022; originally announced June 2022.

Comments: Accepted to NeurIPS 2022

Deepchecks: A Library for Testing and Validating Machine Learning Models and Data

Authors: Shir Chorev, Philip Tannor, Dan Ben Israel, Noam Bressler, Itay Gabbay, Nir Hutnik, Jonatan Liberman, Matan Perlmutter, Yurii Romanyshyn, Lior Rokach

Abstract: This paper presents Deepchecks, a Python library for comprehensively validating machine learning models and data. Our goal is to provide an easy-to-use library comprising of many checks related to various types of issues, such as model predictive performance, data integrity, data distribution mismatches, and more. The package is distributed under the GNU Affero General Public License (AGPL) and re… ▽ More This paper presents Deepchecks, a Python library for comprehensively validating machine learning models and data. Our goal is to provide an easy-to-use library comprising of many checks related to various types of issues, such as model predictive performance, data integrity, data distribution mismatches, and more. The package is distributed under the GNU Affero General Public License (AGPL) and relies on core libraries from the scientific Python ecosystem: scikit-learn, PyTorch, NumPy, pandas, and SciPy. Source code, documentation, examples, and an extensive user guide can be found at \url{https://github.com/deepchecks/deepchecks} and \url{https://docs.deepchecks.com/}. △ Less

Submitted 16 March, 2022; originally announced March 2022.

Overcoming Oversmoothness in Graph Convolutional Networks via Hybrid Scattering Networks

Authors: Frederik Wenkel, Yimeng Min, Matthew Hirn, Michael Perlmutter, Guy Wolf

Abstract: Geometric deep learning has made great strides towards generalizing the design of structure-aware neural networks from traditional domains to non-Euclidean ones, giving rise to graph neural networks (GNN) that can be applied to graph-structured data arising in, e.g., social networks, biochemistry, and material science. Graph convolutional networks (GCNs) in particular, inspired by their Euclidean… ▽ More Geometric deep learning has made great strides towards generalizing the design of structure-aware neural networks from traditional domains to non-Euclidean ones, giving rise to graph neural networks (GNN) that can be applied to graph-structured data arising in, e.g., social networks, biochemistry, and material science. Graph convolutional networks (GCNs) in particular, inspired by their Euclidean counterparts, have been successful in processing graph data by extracting structure-aware features. However, current GNN models are often constrained by various phenomena that limit their expressive power and ability to generalize to more complex graph datasets. Most models essentially rely on low-pass filtering of graph signals via local averaging operations, leading to oversmoothing. Moreover, to avoid severe oversmoothing, most popular GCN-style networks tend to be shallow, with narrow receptive fields, leading to underreaching. Here, we propose a hybrid GNN framework that combines traditional GCN filters with band-pass filters defined via geometric scattering. We further introduce an attention framework that allows the model to locally attend over combined information from different filters at the node level. Our theoretical results establish the complementary benefits of the scattering filters to leverage structural information from the graph, while our experiments show the benefits of our method on various learning tasks. △ Less

Submitted 14 August, 2022; v1 submitted 21 January, 2022; originally announced January 2022.

Comments: This work has been submitted to the IEEE for possible publication

MSC Class: 68T07

Toward Fast and Provably Accurate Near-field Ptychographic Phase Retrieval

Authors: Mark Iwen, Michael Perlmutter, Mark Philip Roach

Abstract: Ptychography is an imaging technique which involves a sample being illuminated by a coherent, localized probe of illumination. When the probe interacts with the sample, the light is diffracted and a diffraction pattern is detected. Then the sample (or probe) is shifted laterally in space to illuminate a new area of the sample whilst ensuring sufficient overlap. Near-field Ptychography (NFP) occurs… ▽ More Ptychography is an imaging technique which involves a sample being illuminated by a coherent, localized probe of illumination. When the probe interacts with the sample, the light is diffracted and a diffraction pattern is detected. Then the sample (or probe) is shifted laterally in space to illuminate a new area of the sample whilst ensuring sufficient overlap. Near-field Ptychography (NFP) occurs when the sample is placed at a short defocus distance having a large Fresnel number. In this paper, we prove that certain NFP measurements are robustly invertible (up to an unavoidable global phase ambiguity) by constructing a point spread function and physical mask which leads to a well-conditioned lifted linear system. We then apply a block phase retrieval algorithm using weighted angular synchronization and prove that the proposed approach accurately recovers the measured sample. Finally, we also propose using a Wirtinger Flow for NFP problems and numerically evaluate that alternate approach both against our main proposed approach, as well as with NFP measurements for which our main approach does not apply. △ Less

Submitted 30 January, 2023; v1 submitted 20 December, 2021; originally announced December 2021.

MSC Class: 65T99; 65Z05; 94A12 ACM Class: G.1.0

Towards a Taxonomy of Graph Learning Datasets

Authors: Renming Liu, Semih Cantürk, Frederik Wenkel, Dylan Sandfelder, Devin Kreuzer, Anna Little, Sarah McGuire, Leslie O'Bray, Michael Perlmutter, Bastian Rieck, Matthew Hirn, Guy Wolf, Ladislav Rampášek

Abstract: Graph neural networks (GNNs) have attracted much attention due to their ability to leverage the intrinsic geometries of the underlying data. Although many different types of GNN models have been developed, with many benchmarking procedures to demonstrate the superiority of one GNN model over the others, there is a lack of systematic understanding of the underlying benchmarking datasets, and what a… ▽ More Graph neural networks (GNNs) have attracted much attention due to their ability to leverage the intrinsic geometries of the underlying data. Although many different types of GNN models have been developed, with many benchmarking procedures to demonstrate the superiority of one GNN model over the others, there is a lack of systematic understanding of the underlying benchmarking datasets, and what aspects of the model are being tested. Here, we provide a principled approach to taxonomize graph benchmarking datasets by carefully designing a collection of graph perturbations to probe the essential data characteristics that GNN models leverage to perform predictions. Our data-driven taxonomization of graph datasets provides a new understanding of critical dataset characteristics that will enable better model evaluation and the development of more specialized GNN models. △ Less

Submitted 27 October, 2021; originally announced October 2021.

Comments: in Data-Centric AI Workshop at NeurIPS 2021

Molecular Graph Generation via Geometric Scattering

Authors: Dhananjay Bhaskar, Jackson D. Grady, Michael A. Perlmutter, Smita Krishnaswamy

Abstract: Graph neural networks (GNNs) have been used extensively for addressing problems in drug design and discovery. Both ligand and target molecules are represented as graphs with node and edge features encoding information about atomic elements and bonds respectively. Although existing deep learning models perform remarkably well at predicting physicochemical properties and binding affinities, the gene… ▽ More Graph neural networks (GNNs) have been used extensively for addressing problems in drug design and discovery. Both ligand and target molecules are represented as graphs with node and edge features encoding information about atomic elements and bonds respectively. Although existing deep learning models perform remarkably well at predicting physicochemical properties and binding affinities, the generation of new molecules with optimized properties remains challenging. Inherently, most GNNs perform poorly in whole-graph representation due to the limitations of the message-passing paradigm. Furthermore, step-by-step graph generation frameworks that use reinforcement learning or other sequential processing can be slow and result in a high proportion of invalid molecules with substantial post-processing needed in order to satisfy the principles of stoichiometry. To address these issues, we propose a representation-first approach to molecular graph generation. We guide the latent representation of an autoencoder by capturing graph structure information with the geometric scattering transform and apply penalties that structure the representation also by molecular properties. We show that this highly structured latent space can be directly used for molecular graph generation by the use of a GAN. We demonstrate that our architecture learns meaningful representations of drug datasets and provides a platform for goal-directed drug synthesis. △ Less

Submitted 12 October, 2021; originally announced October 2021.

A Hybrid Scattering Transform for Signals with Isolated Singularities

Authors: Michael Perlmutter, Jieqian He, Mark Iwen, Matthew Hirn

Abstract: The scattering transform is a wavelet-based model of Convolutional Neural Networks originally introduced by S. Mallat. Mallat's analysis shows that this network has desirable stability and invariance guarantees and therefore helps explain the observation that the filters learned by early layers of a Convolutional Neural Network typically resemble wavelets. Our aim is to understand what sort of fil… ▽ More The scattering transform is a wavelet-based model of Convolutional Neural Networks originally introduced by S. Mallat. Mallat's analysis shows that this network has desirable stability and invariance guarantees and therefore helps explain the observation that the filters learned by early layers of a Convolutional Neural Network typically resemble wavelets. Our aim is to understand what sort of filters should be used in the later layers of the network. Towards this end, we propose a two-layer hybrid scattering transform. In our first layer, we convolve the input signal with a wavelet filter transform to promote sparsity, and, in the second layer, we convolve with a Gabor filter to leverage the sparsity created by the first layer. We show that these measurements characterize information about signals with isolated singularities. We also show that the Gabor measurements used in the second layer can be used to synthesize sparse signals such as those produced by the first layer. △ Less

Submitted 10 October, 2021; originally announced October 2021.

MSC Class: 94A12

On audio enhancement via online non-negative matrix factorization

Authors: Andrew Sack, Wenzhao Jiang, Michael Perlmutter, Palina Salanevich, Deanna Needell

Abstract: We propose a method for noise reduction, the task of producing a clean audio signal from a recording corrupted by additive noise. Many common approaches to this problem are based upon applying non-negative matrix factorization to spectrogram measurements. These methods use a noiseless recording, which is believed to be similar in structure to the signal of interest, and a pure-noise recording to l… ▽ More We propose a method for noise reduction, the task of producing a clean audio signal from a recording corrupted by additive noise. Many common approaches to this problem are based upon applying non-negative matrix factorization to spectrogram measurements. These methods use a noiseless recording, which is believed to be similar in structure to the signal of interest, and a pure-noise recording to learn dictionaries for the true signal and the noise. One may then construct an approximation of the true signal by projecting the corrupted recording on to the clean dictionary. In this work, we build upon these methods by proposing the use of \emph{online} non-negative matrix factorization for this problem. This method is more memory efficient than traditional non-negative matrix factorization and also has potential applications to real-time denoising. △ Less

Submitted 6 October, 2021; originally announced October 2021.

MSC Class: 94A12

Modewise Operators, the Tensor Restricted Isometry Property, and Low-Rank Tensor Recovery

Authors: Mark A. Iwen, Deanna Needell, Michael Perlmutter, Elizaveta Rebrova

Abstract: Recovery of sparse vectors and low-rank matrices from a small number of linear measurements is well-known to be possible under various model assumptions on the measurements. The key requirement on the measurement matrices is typically the restricted isometry property, that is, approximate orthonormality when acting on the subspace to be recovered. Among the most widely used random matrix measureme… ▽ More Recovery of sparse vectors and low-rank matrices from a small number of linear measurements is well-known to be possible under various model assumptions on the measurements. The key requirement on the measurement matrices is typically the restricted isometry property, that is, approximate orthonormality when acting on the subspace to be recovered. Among the most widely used random matrix measurement models are (a) independent sub-gaussian models and (b) randomized Fourier-based models, allowing for the efficient computation of the measurements. For the now ubiquitous tensor data, direct application of the known recovery algorithms to the vectorized or matricized tensor is awkward and memory-heavy because of the huge measurement matrices to be constructed and stored. In this paper, we propose modewise measurement schemes based on sub-gaussian and randomized Fourier measurements. These modewise operators act on the pairs or other small subsets of the tensor modes separately. They require significantly less memory than the measurements working on the vectorized tensor, provably satisfy the tensor restricted isometry property and experimentally can recover the tensor data from fewer measurements and do not require impractical storage. △ Less

Submitted 21 September, 2021; originally announced September 2021.

MSC Class: 15B52; 15A69; 15A83; 97N40

Phase Retrieval for $L^2([-π,π])$ via the Provably Accurate and Noise Robust Numerical Inversion of Spectrogram Measurements

Authors: Mark Iwen, Michael Perlmutter, Nada Sissouno, Aditya Viswanathan

Abstract: In this paper, we focus on the approximation of smooth functions $f: [-π, π] \rightarrow \mathbb{C}$, up to an unresolvable global phase ambiguity, from a finite set of Short Time Fourier Transform (STFT) magnitude (i.e., spectrogram) measurements. Two algorithms are developed for approximately inverting such measurements, each with theoretical error guarantees establishing their correctness. A de… ▽ More In this paper, we focus on the approximation of smooth functions $f: [-π, π] \rightarrow \mathbb{C}$, up to an unresolvable global phase ambiguity, from a finite set of Short Time Fourier Transform (STFT) magnitude (i.e., spectrogram) measurements. Two algorithms are developed for approximately inverting such measurements, each with theoretical error guarantees establishing their correctness. A detailed numerical study also demonstrates that both algorithms work well in practice and have good numerical convergence behavior. △ Less

Submitted 4 June, 2021; originally announced June 2021.

MSC Class: 65T60; 65Z05; 94A12

MagNet: A Neural Network for Directed Graphs

Authors: Xitong Zhang, Yixuan He, Nathan Brugnone, Michael Perlmutter, Matthew Hirn

Abstract: The prevalence of graph-based data has spurred the rapid development of graph neural networks (GNNs) and related machine learning algorithms. Yet, despite the many datasets naturally modeled as directed graphs, including citation, website, and traffic networks, the vast majority of this research focuses on undirected graphs. In this paper, we propose MagNet, a spectral GNN for directed graphs base… ▽ More The prevalence of graph-based data has spurred the rapid development of graph neural networks (GNNs) and related machine learning algorithms. Yet, despite the many datasets naturally modeled as directed graphs, including citation, website, and traffic networks, the vast majority of this research focuses on undirected graphs. In this paper, we propose MagNet, a spectral GNN for directed graphs based on a complex Hermitian matrix known as the magnetic Laplacian. This matrix encodes undirected geometric structure in the magnitude of its entries and directional information in their phase. A "charge" parameter attunes spectral information to variation among directed cycles. We apply our network to a variety of directed graph node classification and link prediction tasks showing that MagNet performs well on all tasks and that its performance exceeds all other methods on a majority of such tasks. The underlying principles of MagNet are such that it can be adapted to other spectral GNN architectures. △ Less

Submitted 11 June, 2021; v1 submitted 22 February, 2021; originally announced February 2021.

Comments: 22 pages, 4 figures, 15 tables. v2: Numerous revisions to the paper's content, including: a more general and informative presentation; many new numerical experiments; revised figures

Understanding Graph Neural Networks with Generalized Geometric Scattering Transforms

Authors: Michael Perlmutter, Alexander Tong, Feng Gao, Guy Wolf, Matthew Hirn

Abstract: The scattering transform is a multilayered wavelet-based deep learning architecture that acts as a model of convolutional neural networks. Recently, several works have introduced generalizations of the scattering transform for non-Euclidean settings such as graphs. Our work builds upon these constructions by introducing windowed and non-windowed geometric scattering transforms for graphs based upo… ▽ More The scattering transform is a multilayered wavelet-based deep learning architecture that acts as a model of convolutional neural networks. Recently, several works have introduced generalizations of the scattering transform for non-Euclidean settings such as graphs. Our work builds upon these constructions by introducing windowed and non-windowed geometric scattering transforms for graphs based upon a very general class of asymmetric wavelets. We show that these asymmetric graph scattering transforms have many of the same theoretical guarantees as their symmetric counterparts. As a result, the proposed construction unifies and extends known theoretical results for many of the existing graph scattering architectures. In doing so, this work helps bridge the gap between geometric scattering and other graph neural networks by introducing a large family of networks with provable stability and invariance guarantees. These results lay the groundwork for future deep learning architectures for graph-structured data that have learned filters and also provably have desirable theoretical properties. △ Less

Submitted 28 June, 2023; v1 submitted 14 November, 2019; originally announced November 2019.

Inverting Spectrogram Measurements via Aliased Wigner Distribution Deconvolution and Angular Synchronization

Authors: Michael Perlmutter, Sami Merhi, Aditya Viswanathan, Mark Iwen

Abstract: We propose a two-step approach for reconstructing a signal ${\bf x}\in\mathbb{C}^d$ from subsampled short-time Fourier transform magnitude (spectogram) measurements: First, we use an aliased Wigner distribution deconvolution approach to solve for a portion of the rank-one matrix ${\bf \widehat{\bf x}}{\bf \widehat{\bf x}}^{*}.$ Second, we use angular syncrhonization to solve for… ▽ More We propose a two-step approach for reconstructing a signal ${\bf x}\in\mathbb{C}^d$ from subsampled short-time Fourier transform magnitude (spectogram) measurements: First, we use an aliased Wigner distribution deconvolution approach to solve for a portion of the rank-one matrix ${\bf \widehat{\bf x}}{\bf \widehat{\bf x}}^{*}.$ Second, we use angular syncrhonization to solve for ${\bf \widehat{\bf x}}$ (and then for ${\bf x}$ by Fourier inversion). Using this method, we produce two new efficient phase retrieval algorithms that perform well numerically in comparison to standard approaches and also prove two theorems, one which guarantees the recovery of discrete, bandlimited signals ${\bf x}\in\mathbb{C}^{d}$ from fewer than $d$ STFT magnitude measurements and another which establishes a new class of deterministic coded diffraction pattern measurements which are guaranteed to allow efficient and noise robust recovery. △ Less

Submitted 24 July, 2019; originally announced July 2019.

MSC Class: 65T60; 65Z05; 94A12

Geometric Wavelet Scattering Networks on Compact Riemannian Manifolds

Authors: Michael Perlmutter, Feng Gao, Guy Wolf, Matthew Hirn

Abstract: The Euclidean scattering transform was introduced nearly a decade ago to improve the mathematical understanding of convolutional neural networks. Inspired by recent interest in geometric deep learning, which aims to generalize convolutional neural networks to manifold and graph-structured domains, we define a geometric scattering transform on manifolds. Similar to the Euclidean scattering transfor… ▽ More The Euclidean scattering transform was introduced nearly a decade ago to improve the mathematical understanding of convolutional neural networks. Inspired by recent interest in geometric deep learning, which aims to generalize convolutional neural networks to manifold and graph-structured domains, we define a geometric scattering transform on manifolds. Similar to the Euclidean scattering transform, the geometric scattering transform is based on a cascade of wavelet filters and pointwise nonlinearities. It is invariant to local isometries and stable to certain types of diffeomorphisms. Empirical results demonstrate its utility on several geometric learning tasks. Our results generalize the deformation stability and local translation invariance of Euclidean scattering, and demonstrate the importance of linking the used filter structures to the underlying geometry of the data. △ Less

Submitted 25 July, 2023; v1 submitted 24 May, 2019; originally announced May 2019.

Comments: 35 pages; 3 figures; 2 tables; v4: Fixed a minor error. Convergence in Equation 13 is in L2 not p.w. modified proof of Theorem 3.3 accordingly

Journal ref: Proceedings of The First Mathematical and Scientific Machine Learning Conference, PMLR 107:570-604, 2020

Scattering Statistics of Generalized Spatial Poisson Point Processes

Authors: Michael Perlmutter, Jieqian He, Matthew Hirn

Abstract: We present a machine learning model for the analysis of randomly generated discrete signals, modeled as the points of an inhomogeneous, compound Poisson point process. Like the wavelet scattering transform introduced by Mallat, our construction is naturally invariant to translations and reflections, but it decouples the roles of scale and frequency, replacing wavelets with Gabor-type measurements.… ▽ More We present a machine learning model for the analysis of randomly generated discrete signals, modeled as the points of an inhomogeneous, compound Poisson point process. Like the wavelet scattering transform introduced by Mallat, our construction is naturally invariant to translations and reflections, but it decouples the roles of scale and frequency, replacing wavelets with Gabor-type measurements. We show that, with suitable nonlinearities, our measurements distinguish Poisson point processes from common self-similar processes, and separate different types of Poisson point processes. △ Less

Submitted 8 October, 2021; v1 submitted 9 February, 2019; originally announced February 2019.

Geometric Scattering on Manifolds

Authors: Michael Perlmutter, Guy Wolf, Matthew Hirn

Abstract: The Euclidean scattering transform was introduced nearly a decade ago to improve the mathematical understanding of the success of convolutional neural networks (ConvNets) in image data analysis and other tasks. Inspired by recent interest in geometric deep learning, which aims to generalize ConvNets to manifold and graph-structured domains, we generalize the scattering transform to compact manifol… ▽ More The Euclidean scattering transform was introduced nearly a decade ago to improve the mathematical understanding of the success of convolutional neural networks (ConvNets) in image data analysis and other tasks. Inspired by recent interest in geometric deep learning, which aims to generalize ConvNets to manifold and graph-structured domains, we generalize the scattering transform to compact manifolds. Similar to the Euclidean scattering transform, our geometric scattering transform is based on a cascade of designed filters and pointwise nonlinearities, which enables rigorous analysis of the feature extraction provided by scattering layers. Our main focus here is on theoretical understanding of this geometric scattering network, while setting aside implementation aspects, although we remark that application of similar transforms to graph data analysis has been studied recently in related work. Our results establish conditions under which geometric scattering provides localized isometry invariant descriptions of manifold signals, which are also stable to families of diffeomorphisms formulated in intrinsic manifolds terms. These results not only generalize the deformation stability and local roto-translation invariance of Euclidean scattering, but also demonstrate the importance of linking the used filter structures (e.g., in geometric deep learning) to the underlying manifold geometry, or the data geometry it represents. △ Less

Submitted 4 June, 2019; v1 submitted 15 December, 2018; originally announced December 2018.

Comments: A shorter version of this paper appeared in the NeurIPS 2018 Integration of Deep Learning Theories Workshop, Montréal, Canada

Feedback Integrators for Nonholonomic Mechanical Systems

Authors: Dong Eui Chang, Matthew Perlmutter

Abstract: The theory of feedback integrators is extended to handle mechanical systems with nonholonomic constraints with or without symmetry, so as to produce numerical integrators that preserve the nonholonomic constraints as well as other conserved quantities. To extend the feedback integrators, we develop a suitable extension theory for nonholonomic systems, and also a corresponding reduction theory for… ▽ More The theory of feedback integrators is extended to handle mechanical systems with nonholonomic constraints with or without symmetry, so as to produce numerical integrators that preserve the nonholonomic constraints as well as other conserved quantities. To extend the feedback integrators, we develop a suitable extension theory for nonholonomic systems, and also a corresponding reduction theory for systems with symmetry. It is then applied to various nonholonomic systems such as the Suslov problem on SO(3), the knife edge, the Chaplygin sleigh, the vertical rolling disk, the roller racer, the Heisenberg system, and the nonholonomic oscillator. △ Less

Submitted 8 November, 2018; originally announced November 2018.

Lower Lipschitz Bounds for Phase Retrieval from Locally Supported Measurements

Authors: Mark A. Iwen, Sami Merhi, Michael Perlmutter

Abstract: In this short note, we consider the worst case noise robustness of any phase retrieval algorithm which aims to reconstruct all nonvanishing vectors $\mathbf{x} \in \mathbb{C}^d$ (up to a single global phase multiple) from the magnitudes of an arbitrary collection of local correlation measurements. Examples of such measurements include both spectrogram measurements of $\mathbf{x}$ using locally sup… ▽ More In this short note, we consider the worst case noise robustness of any phase retrieval algorithm which aims to reconstruct all nonvanishing vectors $\mathbf{x} \in \mathbb{C}^d$ (up to a single global phase multiple) from the magnitudes of an arbitrary collection of local correlation measurements. Examples of such measurements include both spectrogram measurements of $\mathbf{x}$ using locally supported windows and masked Fourier transform intensity measurements of $\mathbf{x}$ using bandlimited masks. As a result, the robustness results considered herein apply to a wide range of both ptychographic and Fourier ptychographic imaging scenarios. In particular, the main results imply that the accurate recovery of high-resolution images of extremely large samples using highly localized probes is likely to require an extremely large number of measurements in order to be robust to worst case measurement noise, independent of the recovery algorithm employed. Furthermore, recent pushes to achieve high-speed and high-resolution ptychographic imaging of integrated circuits for process verification and failure analysis will likely need to carefully balance probe design (e.g., their effective time-frequency support) against the total number of measurements acquired in order for their imaging techniques to be stable to measurement noise, no matter what reconstruction algorithms are applied. △ Less

Submitted 21 June, 2018; originally announced June 2018.

MSC Class: 94A20 (Primary); 94A12 (Secondary); 65Z05 (Secondary); 65T50 (Secondary)

Large Deviations from the Hydrodynamic Limit for a System with Nearest Neighbor Interactions

Authors: Sayan Banerjee, Amarjit Budhiraja, Michael Perlmutter

Abstract: We give a new proof of the large deviation principle from the hydrodynamic limit for the Ginzberg-Landau model studied in Donsker and Varadhan (1989) using techniques from the theory of stochastic control and weak convergence methods. The proof is based on characterizing subsequential hydrodynamic limits of controlled diffusions with nearest neighbor interaction that arise from a variational repre… ▽ More We give a new proof of the large deviation principle from the hydrodynamic limit for the Ginzberg-Landau model studied in Donsker and Varadhan (1989) using techniques from the theory of stochastic control and weak convergence methods. The proof is based on characterizing subsequential hydrodynamic limits of controlled diffusions with nearest neighbor interaction that arise from a variational representation of certain Laplace functionals. The approach taken here does not require superexponential probability estimates, estimation of exponential moments, or an analysis of eigenvalue problems, that are central ingredients in previous proofs. Instead, proof techniques are very similar to those used for the law of large number analysis, namely in the proof of convergence to the hydrodynamic limit (cf. Guo, Papanicolaou and Varadhan (1988)). Specifically, the key step in the proof is establishing suitable bounds on relative entropies and Dirichlet forms associated with certain controlled laws. This general approach has the promise to be applicable to other interacting particle systems as well and to the case of non-equilibrium starting configurations, and to infinite volume systems. △ Less

Submitted 25 March, 2018; originally announced March 2018.

Comments: 48 pages

Feedback Integrators

Authors: Dong Eui Chang, Fernando Jimenez, Matthew Perlmutter

Abstract: A new method is proposed to numerically integrate a dynamical system on a manifold such that the trajectory stably remains on the manifold and preserves first integrals of the system. The idea is that given an initial point in the manifold we extend the dynamics from the manifold to its ambient Euclidean space and then modify the dynamics outside the intersection of the manifold and the level sets… ▽ More A new method is proposed to numerically integrate a dynamical system on a manifold such that the trajectory stably remains on the manifold and preserves first integrals of the system. The idea is that given an initial point in the manifold we extend the dynamics from the manifold to its ambient Euclidean space and then modify the dynamics outside the intersection of the manifold and the level sets of the first integrals containing the initial point such that the intersection becomes a unique local attractor of the resultant dynamics. While the modified dynamics theoretically produces the same trajectory as the original dynamics, it yields a numerical trajectory that stably remains on the manifold and preserves the first integrals. The big merit of our method is that the modified dynamics can be integrated with any ordinary numerical integrator such as Euler or Runge-Kutta. We illustrate this method by applying it to three famous problems: the free rigid body, the Kepler problem and a perturbed Kepler problem with rotational symmetry. We also carry out simulation studies to demonstrate the excellence of our method and make comparisons with the standard projection method, a splitting method and Störmer-Verlet schemes. △ Less

Submitted 15 June, 2016; originally announced June 2016.

Comments: 24 pages, 10 figures

MSC Class: 65P10; 37J99; 93D15; 34C99

A Method of Rotations for Lévy Multipliers

Authors: Michael Perlmutter

Abstract: We use a method of rotations to study the $L^p$ boundedness, $1<p<\infty$, of Fourier multipliers which arise as the projection of martingale transforms with respect to symmetric $α$-stable processes, $0<α<2$. Our proof does not use the fact that $0<α<2$, and therefore allows us to obtain a larger class of multipliers which are bounded on $L^p$. As in the case of the multipliers which arise as the… ▽ More We use a method of rotations to study the $L^p$ boundedness, $1<p<\infty$, of Fourier multipliers which arise as the projection of martingale transforms with respect to symmetric $α$-stable processes, $0<α<2$. Our proof does not use the fact that $0<α<2$, and therefore allows us to obtain a larger class of multipliers which are bounded on $L^p$. As in the case of the multipliers which arise as the projection of martingale transforms, these new multipliers also have potential applications to the study of the $L^p$ boundedness of the Beurling-Ahlfors transform. △ Less

Submitted 14 August, 2015; v1 submitted 13 August, 2015; originally announced August 2015.

MSC Class: 60G46; 42A61

On a class of Calderón-Zygmund operators arising from projections of martingale transforms

Authors: Michael Perlmutter

Abstract: We prove that a large class of operators, which arise as the projections of martingale transforms of stochastic integrals with respect to Brownian motion, as well as other closely related operators, are in fact Calderón--Zygmund operators. Consequently, such operators are not only bounded on $L^p$, $1<p<\infty$, but also satisfy weak-type inequalities. Unlike the boundedness on $L^p,$ which can be… ▽ More We prove that a large class of operators, which arise as the projections of martingale transforms of stochastic integrals with respect to Brownian motion, as well as other closely related operators, are in fact Calderón--Zygmund operators. Consequently, such operators are not only bounded on $L^p$, $1<p<\infty$, but also satisfy weak-type inequalities. Unlike the boundedness on $L^p,$ which can be obtained directly from the Burkholder martingale transform inequalities, the weak-type estimates do not follow from the corresponding martingale results. △ Less

Submitted 22 November, 2013; originally announced November 2013.

MSC Class: 60G52; 45B20

On Hodge decomposition and conformal variational problems

Authors: Stephen Marsland, Robert McLachlan, Klas Modin, Matthew Perlmutter

Abstract: The main result is the identification of the orthogonal complement of the subalgebra of conformal vector field inside the algebra of all vector fields of a compact flat 2-manifold. As a fundamental tool, the complete Hodge decomposition for manifold with boundary is used. The identification allows the derivation of governing differential equations for variational problems on the space of conformal… ▽ More The main result is the identification of the orthogonal complement of the subalgebra of conformal vector field inside the algebra of all vector fields of a compact flat 2-manifold. As a fundamental tool, the complete Hodge decomposition for manifold with boundary is used. The identification allows the derivation of governing differential equations for variational problems on the space of conformal vector fields. Several examples are given. In addition, the paper also gives a review, in full detail, of already known vector field decompositions involving subalgebras of volume preserving and symplectic vector fields. △ Less

Submitted 20 March, 2012; originally announced March 2012.

MSC Class: 58E30; 53D25

Journal ref: J. Phys. A, 47(14):145204, 2014

Geodesic Warps by Conformal Mappings

Authors: Stephen Marsland, Robert McLachlan, Klas Modin, Matthew Perlmutter

Abstract: In recent years there has been considerable interest in methods for diffeomorphic warping of images, with applications e.g.\ in medical imaging and evolutionary biology. The original work generally cited is that of the evolutionary biologist D'Arcy Wentworth Thompson, who demonstrated warps to deform images of one species into another. However, unlike the deformations in modern methods, which are… ▽ More In recent years there has been considerable interest in methods for diffeomorphic warping of images, with applications e.g.\ in medical imaging and evolutionary biology. The original work generally cited is that of the evolutionary biologist D'Arcy Wentworth Thompson, who demonstrated warps to deform images of one species into another. However, unlike the deformations in modern methods, which are drawn from the full set of diffeomorphism, he deliberately chose lower-dimensional sets of transformations, such as planar conformal mappings. In this paper we study warps of such conformal mappings. The approach is to equip the infinite dimensional manifold of conformal embeddings with a Riemannian metric, and then use the corresponding geodesic equation in order to obtain diffeomorphic warps. After deriving the geodesic equation, a numerical discretisation method is developed. Several examples of geodesic warps are then given. We also show that the equation admits totally geodesic solutions corresponding to scaling and translation, but not to affine transformations. △ Less

Submitted 25 July, 2012; v1 submitted 18 March, 2012; originally announced March 2012.

MSC Class: 22E65; 49M99; 58D15

Journal ref: Int. J. Comput. Vis., 2012

On Singular Poisson Sternberg Spaces

Authors: Matthew Perlmutter, Miguel Rodriguez-Olmos

Abstract: We obtain a theory of stratified Sternberg spaces thereby extending the theory of cotangent bundle reduction for free actions to the singular case where the action on the base manifold consists of only one orbit type. We find that the symplectic reduced spaces are stratified topological fiber bundles over the cotangent bundle of the orbit space. We also obtain a Poisson stratification of the Ste… ▽ More We obtain a theory of stratified Sternberg spaces thereby extending the theory of cotangent bundle reduction for free actions to the singular case where the action on the base manifold consists of only one orbit type. We find that the symplectic reduced spaces are stratified topological fiber bundles over the cotangent bundle of the orbit space. We also obtain a Poisson stratification of the Sternberg space. To construct the singular Poisson Sternberg space we develop an appropriate theory of singular connections for proper group actions on a single orbit type manifold including a theory of holonomy extending the usual Ambrose-Singer theorem for principal bundles. △ Less

Submitted 14 March, 2008; originally announced March 2008.

MSC Class: 53E20; 53D17; 57N80

Journal ref: J. Symp. Geom. 7 2 (2009) 15-49

Extensions of Lie-Rinehart algebras and cotangent bundle reduction

Authors: Johannes Huebschmann, Matthew Perlmutter, Tudor S. Ratiu

Abstract: Let Q denote a smooth manifold acted upon smoothly by a Lie group G. The G-action lifts to an action on the total space T of the cotangent bundle of Q and hence on the standard symplectic Poisson algebra of smooth functions on T. The Poisson algebra of G-invariant functions on T yields a Poisson structure on the space T/G of G-orbits. We relate this Poisson algebra with extensions of Lie-Rinehart… ▽ More Let Q denote a smooth manifold acted upon smoothly by a Lie group G. The G-action lifts to an action on the total space T of the cotangent bundle of Q and hence on the standard symplectic Poisson algebra of smooth functions on T. The Poisson algebra of G-invariant functions on T yields a Poisson structure on the space T/G of G-orbits. We relate this Poisson algebra with extensions of Lie-Rinehart algebras and derive an explicit formula for this Poisson structure in terms of differentials. We then show, for the particular case where the G-action on Q is principal, how an explicit description of the Poisson algebra derived in the literature by an ad hoc construction is essentially a special case of the formula for the corresponding extension of Lie-Rinehart algebras. By means of various examples, we also show that this kind of description breaks down when the G-action does not define a principal bundle. △ Less

Submitted 13 March, 2013; v1 submitted 26 January, 2008; originally announced January 2008.

Comments: The original version has been reworked and expanded with coauthors. The new version has 30 pages; it will appear in the Proceedings of the London Mathematical Society

MSC Class: 53D20; 17B63; 17B65; 17B66; 17B81; 22E70; 53D17; 81S10

Journal ref: Proceedings of the London Mathematical Society 107 (2013), 1135-1172

The Symplectic Normal Space of a Cotangent-Lifted Action

Authors: Matthew Perlmutter, Miguel Rodriguez-Olmos, M. Esmeralda Sousa-Dias

Abstract: For the cotangent bundle of a smooth Riemannian manifold acted upon by the lift of a smooth and proper action by isometries of a Lie group, we characterize the symplectic normal space at any point. We show that this space splits as the direct sum of the cotangent bundle of a linear space and a symplectic linear space coming from reduction of a coadjoint orbit. This characterization of the symple… ▽ More For the cotangent bundle of a smooth Riemannian manifold acted upon by the lift of a smooth and proper action by isometries of a Lie group, we characterize the symplectic normal space at any point. We show that this space splits as the direct sum of the cotangent bundle of a linear space and a symplectic linear space coming from reduction of a coadjoint orbit. This characterization of the symplectic normal space can be expressed solely in terms of the group action on the base manifold and the coadjoint representation. Some relevant particular cases are explored. △ Less

Submitted 9 October, 2006; v1 submitted 13 January, 2005; originally announced January 2005.

Comments: Replaced with partially rewritten version. Exposition and results are improved and some mistakes fixed

MSC Class: 53D20; 70H14; 70H33

Journal ref: Diff. Geom. App. 26 3 (2008) 277-297

On the geometry of reduced cotangent bundles at zero momentum

Authors: Matthew Perlmutter, Miguel Rodriguez-Olmos, M. Esmeralda Sousa-Dias

Abstract: We consider the problem of cotangent bundle reduction for non free group actions at zero momentum. We show that in this context the symplectic stratification obtained by Sjamaar and Lerman refines in two ways: (i) each symplectic stratum admits a stratification which we call the secondary stratification with two distinct types of pieces, one of which is open and dense and symplectomorphic to a c… ▽ More We consider the problem of cotangent bundle reduction for non free group actions at zero momentum. We show that in this context the symplectic stratification obtained by Sjamaar and Lerman refines in two ways: (i) each symplectic stratum admits a stratification which we call the secondary stratification with two distinct types of pieces, one of which is open and dense and symplectomorphic to a cotangent bundle; (ii) the reduced space at zero momentum admits a finer stratification than the symplectic one into pieces that are coisotropic in their respective symplectic strata. △ Less

Submitted 7 March, 2006; v1 submitted 28 October, 2003; originally announced October 2003.

Comments: 38 pages, 1 figure

MSC Class: 53D20; 37J15

Journal ref: Journal of Geometry and Physics. 57 (2007) 571-596.

Lie group foliations: Dynamical systems and integrators

Authors: Robert I. McLachlan, Matthew Perlmutter, G. Reinout W. Quispel

Abstract: Foliate systems are those which preserve some (possibly singular) foliation of phase space, such as systems with integrals, systems with continuous symmetries, and skew product systems. We study numerical integrators which also preserve the foliation. The case in which the foliation is given by the orbits of an action of a Lie group has a particularly nice structure, which we study in detail, gi… ▽ More Foliate systems are those which preserve some (possibly singular) foliation of phase space, such as systems with integrals, systems with continuous symmetries, and skew product systems. We study numerical integrators which also preserve the foliation. The case in which the foliation is given by the orbits of an action of a Lie group has a particularly nice structure, which we study in detail, giving conditions under which all foliate vector fields can be written as the sum of a vector field tangent to the orbits and a vector field invariant under the group action. This allows the application of many techniques of geometric integration, including splitting methods and Lie group integrators. △ Less

Submitted 27 September, 2002; originally announced September 2002.

Comments: 20 pages