-
Causal Fourier Analysis on Directed Acyclic Graphs and Posets
Authors:
Bastian Seifert,
Chris Wendler,
Markus Püschel
Abstract:
We present a novel form of Fourier analysis, and associated signal processing concepts, for signals (or data) indexed by edge-weighted directed acyclic graphs (DAGs). This means that our Fourier basis yields an eigendecomposition of a suitable notion of shift and convolution operators that we define. DAGs are the common model to capture causal relationships between data values and in this case our…
▽ More
We present a novel form of Fourier analysis, and associated signal processing concepts, for signals (or data) indexed by edge-weighted directed acyclic graphs (DAGs). This means that our Fourier basis yields an eigendecomposition of a suitable notion of shift and convolution operators that we define. DAGs are the common model to capture causal relationships between data values and in this case our proposed Fourier analysis relates data with its causes under a linearity assumption that we define. The definition of the Fourier transform requires the transitive closure of the weighted DAG for which several forms are possible depending on the interpretation of the edge weights. Examples include level of influence, distance, or pollution distribution. Our framework is different from prior GSP: it is specific to DAGs and leverages, and extends, the classical theory of Moebius inversion from combinatorics. For a prototypical application we consider DAGs modeling dynamic networks in which edges change over time. Specifically, we model the spread of an infection on such a DAG obtained from real-world contact tracing data and learn the infection signal from samples assuming sparsity in the Fourier domain.
△ Less
Submitted 9 August, 2023; v1 submitted 16 September, 2022;
originally announced September 2022.
-
Discrete Signal Processing on Meet/Join Lattices
Authors:
Markus Püschel,
Bastian Seifert,
Chris Wendler
Abstract:
A lattice is a partially ordered set supporting a meet (or join) operation that returns the largest lower bound (smallest upper bound) of two elements. Just like graphs, lattices are a fundamental structure that occurs across domains including social data analysis, natural language processing, computational chemistry and biology, and database theory. In this paper we introduce discrete-lattice sig…
▽ More
A lattice is a partially ordered set supporting a meet (or join) operation that returns the largest lower bound (smallest upper bound) of two elements. Just like graphs, lattices are a fundamental structure that occurs across domains including social data analysis, natural language processing, computational chemistry and biology, and database theory. In this paper we introduce discrete-lattice signal processing (DLSP), an SP framework for data, or signals, indexed by such lattices. We use the meet (or join) to define a shift operation and derive associated notions of filtering, Fourier basis and transform, and frequency response. We show that the spectrum of a lattice signal inherits the lattice structure of the signal domain and derive a sampling theorem. Finally, we show two prototypical applications: spectral analysis of formal concept lattices in social science and sampling and Wiener filtering of multiset lattices in combinatorial auctions. Formal concept lattices are a compressed representation of relations between objects and attributes. Since relations are equivalent to bipartite graphs and hypergraphs, DLSP offers a form of Fourier analysis for these structures.
△ Less
Submitted 6 July, 2021; v1 submitted 8 December, 2020;
originally announced December 2020.
-
Learning Set Functions that are Sparse in Non-Orthogonal Fourier Bases
Authors:
Chris Wendler,
Andisheh Amrollahi,
Bastian Seifert,
Andreas Krause,
Markus Püschel
Abstract:
Many applications of machine learning on discrete domains, such as learning preference functions in recommender systems or auctions, can be reduced to estimating a set function that is sparse in the Fourier domain. In this work, we present a new family of algorithms for learning Fourier-sparse set functions. They require at most $nk - k \log_2 k + k$ queries (set function evaluations), under mild…
▽ More
Many applications of machine learning on discrete domains, such as learning preference functions in recommender systems or auctions, can be reduced to estimating a set function that is sparse in the Fourier domain. In this work, we present a new family of algorithms for learning Fourier-sparse set functions. They require at most $nk - k \log_2 k + k$ queries (set function evaluations), under mild conditions on the Fourier coefficients, where $n$ is the size of the ground set and $k$ the number of non-zero Fourier coefficients. In contrast to other work that focused on the orthogonal Walsh-Hadamard transform, our novel algorithms operate with recently introduced non-orthogonal Fourier transforms that offer different notions of Fourier-sparsity. These naturally arise when modeling, e.g., sets of items forming substitutes and complements. We demonstrate effectiveness on several real-world applications.
△ Less
Submitted 29 March, 2021; v1 submitted 1 October, 2020;
originally announced October 2020.
-
Digraph Signal Processing with Generalized Boundary Conditions
Authors:
Bastian Seifert,
Markus Püschel
Abstract:
Signal processing on directed graphs (digraphs) is problematic, since the graph shift, and thus associated filters, are in general not diagonalizable. Furthermore, the Fourier transform in this case is now obtained from the Jordan decomposition, which may not be computable at all for large graphs. We propose a novel and general solution for this problem based on matrix perturbation theory: We desi…
▽ More
Signal processing on directed graphs (digraphs) is problematic, since the graph shift, and thus associated filters, are in general not diagonalizable. Furthermore, the Fourier transform in this case is now obtained from the Jordan decomposition, which may not be computable at all for large graphs. We propose a novel and general solution for this problem based on matrix perturbation theory: We design an algorithm that adds a small number of edges to a given digraph to destroy nontrivial Jordan blocks. The obtained digraph is then diagonalizable and yields, as we show, an approximate eigenbasis and Fourier transform for the original digraph. We explain why and how this construction can be viewed as generalized form of boundary conditions, a common practice in signal processing. Our experiments with random and real world graphs show that we can scale to graphs with a few thousands nodes, and obtain Fourier transforms that are close to orthogonal while still diagonalizing an intuitive notion of convolution. Our method works with adjacency and Laplacian shift and can be used as preprocessing step to enable further processing as we show with a prototypical Wiener filter application.
△ Less
Submitted 8 February, 2021; v1 submitted 19 May, 2020;
originally announced May 2020.
-
Discrete Signal Processing with Set Functions
Authors:
Markus Püschel,
Chris Wendler
Abstract:
Set functions are functions (or signals) indexed by the powerset (set of all subsets) of a finite set N. They are fundamental and ubiquitous in many application domains and have been used, for example, to formally describe or quantify loss functions for semantic image segmentation, the informativeness of sensors in sensor networks the utility of sets of items in recommender systems, cooperative ga…
▽ More
Set functions are functions (or signals) indexed by the powerset (set of all subsets) of a finite set N. They are fundamental and ubiquitous in many application domains and have been used, for example, to formally describe or quantify loss functions for semantic image segmentation, the informativeness of sensors in sensor networks the utility of sets of items in recommender systems, cooperative games in game theory, or bidders in combinatorial auctions. In particular, the subclass of submodular functions occurs in many optimization and machine learning problems. In this paper, we derive discrete-set signal processing (SP), a novel shift-invariant linear signal processing framework for set functions. Discrete-set SP considers different notions of shift obtained from set union and difference operations. For each shift it provides associated notions of shift-invariant filters, convolution, Fourier transform, and frequency response. We provide intuition for our framework using the concept of generalized coverage function that we define, identify multivariate mutual information as a special case of a discrete-set spectrum, and motivate frequency ordering. Our work brings a new set of tools for analyzing and processing set functions, and, in particular, for dealing with their exponential nature. We show two prototypical applications and experiments: compression in submodular function optimization and sampling for preference elicitation in combinatorial auctions.
△ Less
Submitted 22 October, 2020; v1 submitted 28 January, 2020;
originally announced January 2020.
-
Distributed Basis Pursuit
Authors:
João F. C. Mota,
João M. F. Xavier,
Pedro M. Q. Aguiar,
Markus Püschel
Abstract:
We propose a distributed algorithm for solving the optimization problem Basis Pursuit (BP). BP finds the least L1-norm solution of the underdetermined linear system Ax = b and is used, for example, in compressed sensing for reconstruction. Our algorithm solves BP on a distributed platform such as a sensor network, and is designed to minimize the communication between nodes. The algorithm only requ…
▽ More
We propose a distributed algorithm for solving the optimization problem Basis Pursuit (BP). BP finds the least L1-norm solution of the underdetermined linear system Ax = b and is used, for example, in compressed sensing for reconstruction. Our algorithm solves BP on a distributed platform such as a sensor network, and is designed to minimize the communication between nodes. The algorithm only requires the network to be connected, has no notion of a central processing node, and no node has access to the entire matrix A at any time. We consider two scenarios in which either the columns or the rows of A are distributed among the compute nodes. Our algorithm, named D-ADMM, is a decentralized implementation of the alternating direction method of multipliers. We show through numerical simulation that our algorithm requires considerably less communications between the nodes than the state-of-the-art algorithms.
△ Less
Submitted 14 March, 2012; v1 submitted 6 September, 2010;
originally announced September 2010.
