-
Introduction to Nonlinear Spectral Analysis
Authors:
Leon Bungert,
Yury Korolev
Abstract:
These notes are meant as an introduction to the theory of nonlinear spectral theory. We will discuss the variational form of nonlninear eigenvalue problems and the corresponding non-linear Euler--Lagrange equations, as well as connections with gradient flows. For the latter ones, we will give precise conditions for finite time extinction and discuss convergence rates. We will use this theory to st…
▽ More
These notes are meant as an introduction to the theory of nonlinear spectral theory. We will discuss the variational form of nonlninear eigenvalue problems and the corresponding non-linear Euler--Lagrange equations, as well as connections with gradient flows. For the latter ones, we will give precise conditions for finite time extinction and discuss convergence rates. We will use this theory to study asymptotic behaviour of nonlinear PDEs and present applications in $L^\infty$ variational problems. Finally we will discuss numerical methods for solving gradient flows and computing nonlinear eigenfunctions based on a nonlinear power method. Our main tools are convex analysis and calculus of variations, necessary background on which will be provided. It is expected that the reader is familiar with Hilbert spaces; familiarity with Banach spaces is beneficial but not strictly necessary. The notes are based on the lectures taught by the authors at the universities of Bonn and Cambridge in 2022.
△ Less
Submitted 10 June, 2025;
originally announced June 2025.
-
Meshless Shape Optimization using Neural Networks and Partial Differential Equations on Graphs
Authors:
Eloi Martinet,
Leon Bungert
Abstract:
Shape optimization involves the minimization of a cost function defined over a set of shapes, often governed by a partial differential equation (PDE). In the absence of closed-form solutions, one relies on numerical methods to approximate the solution. The level set method -- when coupled with the finite element method -- is one of the most versatile numerical shape optimization approaches but sti…
▽ More
Shape optimization involves the minimization of a cost function defined over a set of shapes, often governed by a partial differential equation (PDE). In the absence of closed-form solutions, one relies on numerical methods to approximate the solution. The level set method -- when coupled with the finite element method -- is one of the most versatile numerical shape optimization approaches but still suffers from the limitations of most mesh-based methods. In this work, we present a fully meshless level set framework that leverages neural networks to parameterize the level set function and employs the graph Laplacian to approximate the underlying PDE. Our approach enables precise computations of geometric quantities such as surface normals and curvature, and allows tackling optimization problems within the class of convex shapes.
△ Less
Submitted 20 February, 2025;
originally announced February 2025.
-
MirrorCBO: A consensus-based optimization method in the spirit of mirror descent
Authors:
Leon Bungert,
Franca Hoffmann,
Doh Yeon Kim,
Tim Roith
Abstract:
In this work we propose MirrorCBO, a consensus-based optimization (CBO) method which generalizes standard CBO in the same way that mirror descent generalizes gradient descent. For this we apply the CBO methodology to a swarm of dual particles and retain the primal particle positions by applying the inverse of the mirror map, which we parametrize as the subdifferential of a strongly convex function…
▽ More
In this work we propose MirrorCBO, a consensus-based optimization (CBO) method which generalizes standard CBO in the same way that mirror descent generalizes gradient descent. For this we apply the CBO methodology to a swarm of dual particles and retain the primal particle positions by applying the inverse of the mirror map, which we parametrize as the subdifferential of a strongly convex function $φ$. In this way, we combine the advantages of a derivative-free non-convex optimization algorithm with those of mirror descent. As a special case, the method extends CBO to optimization problems with convex constraints. Assuming bounds on the Bregman distance associated to $φ$, we provide asymptotic convergence results for MirrorCBO with explicit exponential rate. Another key contribution is an exploratory numerical study of this new algorithm across different application settings, focusing on (i) sparsity-inducing optimization, and (ii) constrained optimization, demonstrating the competitive performance of MirrorCBO. We observe empirically that the method can also be used for optimization on (non-convex) submanifolds of Euclidean space, can be adapted to mirrored versions of other recent CBO variants, and that it inherits from mirror descent the capability to select desirable minimizers, like sparse ones. We also include an overview of recent CBO approaches for constrained optimization and compare their performance to MirrorCBO.
△ Less
Submitted 21 January, 2025;
originally announced January 2025.
-
Convergence rates of the fractional to the local Dirichlet problem
Authors:
Leon Bungert,
Félix del Teso
Abstract:
We prove non-asymptotic rates of convergence in the $W^{s,2}(\mathbb R^d)$-norm for the solution of the fractional Dirichlet problem to the solution of the local Dirichlet problem as $s\uparrow 1$. For regular enough boundary values we get a rate of order $\sqrt{1-s}$, while for less regular data the rate is of order $\sqrt{(1-s)|\log(1-s)|}$. We also obtain results when the right hand side depend…
▽ More
We prove non-asymptotic rates of convergence in the $W^{s,2}(\mathbb R^d)$-norm for the solution of the fractional Dirichlet problem to the solution of the local Dirichlet problem as $s\uparrow 1$. For regular enough boundary values we get a rate of order $\sqrt{1-s}$, while for less regular data the rate is of order $\sqrt{(1-s)|\log(1-s)|}$. We also obtain results when the right hand side depends on $s$, and our error estimates are true for all $s\in(0,1)$. The proofs use variational arguments to deduce rates in the fractional Sobolev norm from energy estimates between the fractional and the standard Dirichlet energy.
△ Less
Submitted 6 August, 2024;
originally announced August 2024.
-
Convergence rates for Poisson learning to a Poisson equation with measure data
Authors:
Leon Bungert,
Jeff Calder,
Max Mihailescu,
Kodjo Houssou,
Amber Yuan
Abstract:
In this paper we prove discrete to continuum convergence rates for Poisson Learning, a graph-based semi-supervised learning algorithm that is based on solving the graph Poisson equation with a source term consisting of a linear combination of Dirac deltas located at labeled points and carrying label information. The corresponding continuum equation is a Poisson equation with measure data in a Eucl…
▽ More
In this paper we prove discrete to continuum convergence rates for Poisson Learning, a graph-based semi-supervised learning algorithm that is based on solving the graph Poisson equation with a source term consisting of a linear combination of Dirac deltas located at labeled points and carrying label information. The corresponding continuum equation is a Poisson equation with measure data in a Euclidean domain $Ω\subset \mathbb{R}^d$. The singular nature of these equations is challenging and requires an approach with several distinct parts: (1) We prove quantitative error estimates when convolving the measure data of a Poisson equation with (approximately) radial function supported on balls. (2) We use quantitative variational techniques to prove discrete to continuum convergence rates on random geometric graphs with bandwidth $\varepsilon>0$ for bounded source terms. (3) We show how to regularize the graph Poisson equation via mollification with the graph heat kernel, and we study fine asymptotics of the heat kernel on random geometric graphs. Combining these three pillars we obtain $L^1$ convergence rates that scale, up to logarithmic factors, like $O(\varepsilon^{\frac{1}{d+2}})$ for general data distributions, and $O(\varepsilon^{\frac{2-σ}{d+4}})$ for uniformly distributed data, where $σ>0$. These rates are valid with high probability if $\varepsilon\gg\left({\log n}/{n}\right)^q$ where $n$ denotes the number of vertices of the graph and $q \approx \frac{1}{3d}$.
△ Less
Submitted 9 July, 2024;
originally announced July 2024.
-
A mean curvature flow arising in adversarial training
Authors:
Leon Bungert,
Tim Laux,
Kerrek Stinson
Abstract:
We connect adversarial training for binary classification to a geometric evolution equation for the decision boundary. Relying on a perspective that recasts adversarial training as a regularization problem, we introduce a modified training scheme that constitutes a minimizing movements scheme for a nonlocal perimeter functional. We prove that the scheme is monotone and consistent as the adversaria…
▽ More
We connect adversarial training for binary classification to a geometric evolution equation for the decision boundary. Relying on a perspective that recasts adversarial training as a regularization problem, we introduce a modified training scheme that constitutes a minimizing movements scheme for a nonlocal perimeter functional. We prove that the scheme is monotone and consistent as the adversarial budget vanishes and the perimeter localizes, and as a consequence we rigorously show that the scheme approximates a weighted mean curvature flow. This highlights that the efficacy of adversarial training may be due to locally minimizing the length of the decision boundary. In our analysis, we introduce a variety of tools for working with the subdifferential of a supremal-type nonlocal total variation and its regularity properties.
△ Less
Submitted 22 April, 2024;
originally announced April 2024.
-
It begins with a boundary: A geometric view on probabilistically robust learning
Authors:
Leon Bungert,
Nicolás García Trillos,
Matt Jacobs,
Daniel McKenzie,
Đorđe Nikolić,
Qingsong Wang
Abstract:
Although deep neural networks have achieved super-human performance on many classification tasks, they often exhibit a worrying lack of robustness towards adversarially generated examples. Thus, considerable effort has been invested into reformulating standard Risk Minimization (RM) into an adversarially robust framework. Recently, attention has shifted towards approaches which interpolate between…
▽ More
Although deep neural networks have achieved super-human performance on many classification tasks, they often exhibit a worrying lack of robustness towards adversarially generated examples. Thus, considerable effort has been invested into reformulating standard Risk Minimization (RM) into an adversarially robust framework. Recently, attention has shifted towards approaches which interpolate between the robustness offered by adversarial training and the higher clean accuracy and faster training times of RM. In this paper, we take a fresh and geometric view on one such method -- Probabilistically Robust Learning (PRL). We propose a mathematical framework for understanding PRL, which allows us to identify geometric pathologies in its original formulation and to introduce a family of probabilistic nonlocal perimeter functionals to rectify them. We prove existence of solutions to the original and modified problems using novel relaxation methods and also study properties, as well as local limits, of the introduced perimeters. We also clarify, through a suitable $Γ$-convergence analysis, the way in which the original and modified PRL models interpolate between risk minimization and adversarial training.
△ Less
Submitted 30 September, 2024; v1 submitted 30 May, 2023;
originally announced May 2023.
-
The convergence rate of $p$-harmonic to infinity-harmonic functions
Authors:
Leon Bungert
Abstract:
The purpose of this paper is to prove a uniform convergence rate of the solutions of the $p$-Laplace equation $Δ_p u = 0$ with Dirichlet boundary conditions to the solution of the infinity-Laplace equation $Δ_\infty u = 0$ as $p\to\infty$. The rate scales like $p^{-1/4}$ for general solutions of the Dirichlet problem and like $p^{-1/2}$ for solutions with positive gradient. An explicit example sho…
▽ More
The purpose of this paper is to prove a uniform convergence rate of the solutions of the $p$-Laplace equation $Δ_p u = 0$ with Dirichlet boundary conditions to the solution of the infinity-Laplace equation $Δ_\infty u = 0$ as $p\to\infty$. The rate scales like $p^{-1/4}$ for general solutions of the Dirichlet problem and like $p^{-1/2}$ for solutions with positive gradient. An explicit example shows that it cannot be better than $p^{-1}$. The proof of this result solely relies on the comparison principle with the fundamental solutions of the $p$-Laplace and the infinity-Laplace equation, respectively. Our argument does not use viscosity solutions, is purely metric, and is therefore generalizable to more general settings where a comparison principle with Hölder cones and Hölder regularity is available.
△ Less
Submitted 10 October, 2023; v1 submitted 16 February, 2023;
originally announced February 2023.
-
Gamma-convergence of a nonlocal perimeter arising in adversarial machine learning
Authors:
Leon Bungert,
Kerrek Stinson
Abstract:
In this paper we prove Gamma-convergence of a nonlocal perimeter of Minkowski type to a local anisotropic perimeter. The nonlocal model describes the regularizing effect of adversarial training in binary classifications. The energy essentially depends on the interaction between two distributions modelling likelihoods for the associated classes. We overcome typical strict regularity assumptions for…
▽ More
In this paper we prove Gamma-convergence of a nonlocal perimeter of Minkowski type to a local anisotropic perimeter. The nonlocal model describes the regularizing effect of adversarial training in binary classifications. The energy essentially depends on the interaction between two distributions modelling likelihoods for the associated classes. We overcome typical strict regularity assumptions for the distributions by only assuming that they have bounded $BV$ densities. In the natural topology coming from compactness, we prove Gamma-convergence to a weighted perimeter with weight determined by an anisotropic function of the two densities. Despite being local, this sharp interface limit reflects classification stability with respect to adversarial perturbations. We further apply our results to deduce Gamma-convergence of the associated total variations, to study the asymptotics of adversarial training, and to prove Gamma-convergence of graph discretizations for the nonlocal perimeter.
△ Less
Submitted 29 January, 2024; v1 submitted 28 November, 2022;
originally announced November 2022.
-
Polarized consensus-based dynamics for optimization and sampling
Authors:
Leon Bungert,
Tim Roith,
Philipp Wacker
Abstract:
In this paper we propose polarized consensus-based dynamics in order to make consensus-based optimization (CBO) and sampling (CBS) applicable for objective functions with several global minima or distributions with many modes, respectively. For this, we ``polarize'' the dynamics with a localizing kernel and the resulting model can be viewed as a bounded confidence model for opinion formation in th…
▽ More
In this paper we propose polarized consensus-based dynamics in order to make consensus-based optimization (CBO) and sampling (CBS) applicable for objective functions with several global minima or distributions with many modes, respectively. For this, we ``polarize'' the dynamics with a localizing kernel and the resulting model can be viewed as a bounded confidence model for opinion formation in the presence of common objective. Instead of being attracted to a common weighted mean as in the original consensus-based methods, which prevents the detection of more than one minimum or mode, in our method every particle is attracted to a weighted mean which gives more weight to nearby particles. We prove that in the mean-field regime the polarized CBS dynamics are unbiased for Gaussian targets. We also prove that in the zero temperature limit and for sufficiently well-behaved strongly convex objectives the solution of the Fokker--Planck equation converges in the Wasserstein-2 distance to a Dirac measure at the minimizer. Finally, we propose a computationally more efficient generalization which works with a predefined number of clusters and improves upon our polarized baseline method for high-dimensional optimization.
△ Less
Submitted 9 October, 2023; v1 submitted 9 November, 2022;
originally announced November 2022.
-
Ratio convergence rates for Euclidean first-passage percolation: Applications to the graph infinity Laplacian
Authors:
Leon Bungert,
Jeff Calder,
Tim Roith
Abstract:
In this paper we prove the first quantitative convergence rates for the graph infinity Laplace equation for length scales at the connectivity threshold. In the graph-based semi-supervised learning community this equation is also known as Lipschitz learning. The graph infinity Laplace equation is characterized by the metric on the underlying space, and convergence rates follow from convergence rate…
▽ More
In this paper we prove the first quantitative convergence rates for the graph infinity Laplace equation for length scales at the connectivity threshold. In the graph-based semi-supervised learning community this equation is also known as Lipschitz learning. The graph infinity Laplace equation is characterized by the metric on the underlying space, and convergence rates follow from convergence rates for graph distances. At the connectivity threshold, this problem is related to Euclidean first passage percolation, which is concerned with the Euclidean distance function $d_{h}(x,y)$ on a homogeneous Poisson point process on $\mathbb{R}^d$, where admissible paths have step size at most $h>0$. Using a suitable regularization of the distance function and subadditivity we prove that ${d_{h_s}(0,se_1)}/ s \to σ$ as $s\to\infty$ almost surely where $σ\geq 1$ is a dimensional constant and $h_s\gtrsim \log(s)^\frac{1}{d}$. A convergence rate is not available due to a lack of approximate superadditivity when $h_s\to \infty$. Instead, we prove convergence rates for the ratio $\frac{d_{h}(0,se_1)}{d_{h}(0,2se_1)}\to \frac{1}{2}$ when $h$ is frozen and does not depend on $s$. Combining this with the techniques that we developed in (Bungert, Calder, Roith, IMA Journal of Numerical Analysis, 2022), we show that this notion of ratio convergence is sufficient to establish uniform convergence rates for solutions of the graph infinity Laplace equation at percolation length scales.
△ Less
Submitted 22 February, 2024; v1 submitted 17 October, 2022;
originally announced October 2022.
-
Improving Robustness against Real-World and Worst-Case Distribution Shifts through Decision Region Quantification
Authors:
Leo Schwinn,
Leon Bungert,
An Nguyen,
René Raab,
Falk Pulsmeyer,
Doina Precup,
Björn Eskofier,
Dario Zanca
Abstract:
The reliability of neural networks is essential for their use in safety-critical applications. Existing approaches generally aim at improving the robustness of neural networks to either real-world distribution shifts (e.g., common corruptions and perturbations, spatial transformations, and natural adversarial examples) or worst-case distribution shifts (e.g., optimized adversarial examples). In th…
▽ More
The reliability of neural networks is essential for their use in safety-critical applications. Existing approaches generally aim at improving the robustness of neural networks to either real-world distribution shifts (e.g., common corruptions and perturbations, spatial transformations, and natural adversarial examples) or worst-case distribution shifts (e.g., optimized adversarial examples). In this work, we propose the Decision Region Quantification (DRQ) algorithm to improve the robustness of any differentiable pre-trained model against both real-world and worst-case distribution shifts in the data. DRQ analyzes the robustness of local decision regions in the vicinity of a given data point to make more reliable predictions. We theoretically motivate the DRQ algorithm by showing that it effectively smooths spurious local extrema in the decision surface. Furthermore, we propose an implementation using targeted and untargeted adversarial attacks. An extensive empirical evaluation shows that DRQ increases the robustness of adversarially and non-adversarially trained models against real-world and worst-case distribution shifts on several computer vision benchmark datasets.
△ Less
Submitted 19 May, 2022;
originally announced May 2022.
-
The inhomogeneous $p$-Laplacian equation with Neumann boundary conditions in the limit $p\to\infty$
Authors:
Leon Bungert
Abstract:
We investigate the limiting behavior of solutions to the inhomogeneous $p$-Laplacian equation $-Δ_p u = μ_p$ subject to Neumann boundary conditions. For right hand sides which are arbitrary signed measures we show that solutions converge to a Kantorovich potential associated with the geodesic Wasserstein-$1$ distance. In the regular case with continuous right hand sides we characterize the limit a…
▽ More
We investigate the limiting behavior of solutions to the inhomogeneous $p$-Laplacian equation $-Δ_p u = μ_p$ subject to Neumann boundary conditions. For right hand sides which are arbitrary signed measures we show that solutions converge to a Kantorovich potential associated with the geodesic Wasserstein-$1$ distance. In the regular case with continuous right hand sides we characterize the limit as viscosity solution to an infinity Laplacian / eikonal type equation.
△ Less
Submitted 13 December, 2022; v1 submitted 14 December, 2021;
originally announced December 2021.
-
The Geometry of Adversarial Training in Binary Classification
Authors:
Leon Bungert,
Nicolás García Trillos,
Ryan Murray
Abstract:
We establish an equivalence between a family of adversarial training problems for non-parametric binary classification and a family of regularized risk minimization problems where the regularizer is a nonlocal perimeter functional. The resulting regularized risk minimization problems admit exact convex relaxations of the type $L^1+$ (nonlocal) $\operatorname{TV}$, a form frequently studied in imag…
▽ More
We establish an equivalence between a family of adversarial training problems for non-parametric binary classification and a family of regularized risk minimization problems where the regularizer is a nonlocal perimeter functional. The resulting regularized risk minimization problems admit exact convex relaxations of the type $L^1+$ (nonlocal) $\operatorname{TV}$, a form frequently studied in image analysis and graph-based learning. A rich geometric structure is revealed by this reformulation which in turn allows us to establish a series of properties of optimal solutions of the original problem, including the existence of minimal and maximal solutions (interpreted in a suitable sense), and the existence of regular solutions (also interpreted in a suitable sense). In addition, we highlight how the connection between adversarial training and perimeter minimization problems provides a novel, directly interpretable, statistical motivation for a family of regularized risk minimization problems involving perimeter/total variation. The majority of our theoretical results are independent of the distance used to define adversarial attacks.
△ Less
Submitted 1 August, 2022; v1 submitted 26 November, 2021;
originally announced November 2021.
-
Uniform Convergence Rates for Lipschitz Learning on Graphs
Authors:
Leon Bungert,
Jeff Calder,
Tim Roith
Abstract:
Lipschitz learning is a graph-based semi-supervised learning method where one extends labels from a labeled to an unlabeled data set by solving the infinity Laplace equation on a weighted graph. In this work we prove uniform convergence rates for solutions of the graph infinity Laplace equation as the number of vertices grows to infinity. Their continuum limits are absolutely minimizing Lipschitz…
▽ More
Lipschitz learning is a graph-based semi-supervised learning method where one extends labels from a labeled to an unlabeled data set by solving the infinity Laplace equation on a weighted graph. In this work we prove uniform convergence rates for solutions of the graph infinity Laplace equation as the number of vertices grows to infinity. Their continuum limits are absolutely minimizing Lipschitz extensions with respect to the geodesic metric of the domain where the graph vertices are sampled from. We work under very general assumptions on the graph weights, the set of labeled vertices, and the continuum domain. Our main contribution is that we obtain quantitative convergence rates even for very sparsely connected graphs, as they typically appear in applications like semi-supervised learning. In particular, our framework allows for graph bandwidths down to the connectivity radius. For proving this we first show a quantitative convergence statement for graph distance functions to geodesic distance functions in the continuum. Using the "comparison with distance functions" principle, we can pass these convergence statements to infinity harmonic functions and absolutely minimizing Lipschitz extensions.
△ Less
Submitted 29 June, 2022; v1 submitted 24 November, 2021;
originally announced November 2021.
-
Eigenvalue Problems in $\mathrm{L}^\infty$: Optimality Conditions, Duality, and Relations with Optimal Transport
Authors:
Leon Bungert,
Yury Korolev
Abstract:
In this article we characterize the $\mathrm{L}^\infty$ eigenvalue problem associated to the Rayleigh quotient $\left.{\|\nabla u\|_{\mathrm{L}^\infty}}\middle/{\|u\|_\infty}\right.$ and relate it to a divergence-form PDE, similarly to what is known for $\mathrm{L}^p$ eigenvalue problems and the $p$-Laplacian for $p<\infty$. Contrary to existing methods, which study $\mathrm{L}^\infty$-problems as…
▽ More
In this article we characterize the $\mathrm{L}^\infty$ eigenvalue problem associated to the Rayleigh quotient $\left.{\|\nabla u\|_{\mathrm{L}^\infty}}\middle/{\|u\|_\infty}\right.$ and relate it to a divergence-form PDE, similarly to what is known for $\mathrm{L}^p$ eigenvalue problems and the $p$-Laplacian for $p<\infty$. Contrary to existing methods, which study $\mathrm{L}^\infty$-problems as limits of $\mathrm{L}^p$-problems for $p\to\infty$, we develop a novel framework for analyzing the limiting problem directly using convex analysis and geometric measure theory. For this, we derive a novel fine characterization of the subdifferential of the Lipschitz-constant-functional $u\mapsto\|\nabla u\|_{\mathrm{L}^\infty}$. We show that the eigenvalue problem takes the form $λνu =-\operatorname{div}(τ\nabla_τu)$, where $ν$ and $τ$ are non-negative measures concentrated where $|u|$ respectively $|\nabla u|$ are maximal, and $\nabla_τu$ is the tangential gradient of $u$ with respect to $τ$. Lastly, we investigate a dual Rayleigh quotient whose minimizers solve an optimal transport problem associated to a generalized Kantorovich--Rubinstein norm. Our results apply to all stationary points of the Rayleigh quotient, including infinity ground states, infinity harmonic potentials, distance functions, etc., and generalize known results in the literature.
△ Less
Submitted 21 October, 2022; v1 submitted 26 July, 2021;
originally announced July 2021.
-
Neural Architecture Search via Bregman Iterations
Authors:
Leon Bungert,
Tim Roith,
Daniel Tenbrinck,
Martin Burger
Abstract:
We propose a novel strategy for Neural Architecture Search (NAS) based on Bregman iterations. Starting from a sparse neural network our gradient-based one-shot algorithm gradually adds relevant parameters in an inverse scale space manner. This allows the network to choose the best architecture in the search space which makes it well-designed for a given task, e.g., by adding neurons or skip connec…
▽ More
We propose a novel strategy for Neural Architecture Search (NAS) based on Bregman iterations. Starting from a sparse neural network our gradient-based one-shot algorithm gradually adds relevant parameters in an inverse scale space manner. This allows the network to choose the best architecture in the search space which makes it well-designed for a given task, e.g., by adding neurons or skip connections. We demonstrate that using our approach one can unveil, for instance, residual autoencoders for denoising, deblurring, and classification tasks. Code is available at https://github.com/TimRoith/BregmanLearning.
△ Less
Submitted 4 June, 2021;
originally announced June 2021.
-
Gradient Flows and Nonlinear Power Methods for the Computation of Nonlinear Eigenfunctions
Authors:
Leon Bungert,
Martin Burger
Abstract:
This chapter describes how gradient flows and nonlinear power methods in Banach spaces can be used to solve nonlinear eigenvector-dependent eigenvalue problems, and how convergence of (discretized) approximations can be verified. We review several flows from literature, which were proposed to compute nonlinear eigenfunctions, and show that they all relate to normalized gradient flows. Furthermore,…
▽ More
This chapter describes how gradient flows and nonlinear power methods in Banach spaces can be used to solve nonlinear eigenvector-dependent eigenvalue problems, and how convergence of (discretized) approximations can be verified. We review several flows from literature, which were proposed to compute nonlinear eigenfunctions, and show that they all relate to normalized gradient flows. Furthermore, we show that the implicit Euler discretization of gradient flows gives rise to a nonlinear power method of the proximal operator and prove their convergence to nonlinear eigenfunctions. Finally, we prove that $Γ$-convergence of functionals implies convergence of their ground states, which is important for discrete approximations.
△ Less
Submitted 8 October, 2021; v1 submitted 18 May, 2021;
originally announced May 2021.
-
A Bregman Learning Framework for Sparse Neural Networks
Authors:
Leon Bungert,
Tim Roith,
Daniel Tenbrinck,
Martin Burger
Abstract:
We propose a learning framework based on stochastic Bregman iterations, also known as mirror descent, to train sparse neural networks with an inverse scale space approach. We derive a baseline algorithm called LinBreg, an accelerated version using momentum, and AdaBreg, which is a Bregmanized generalization of the Adam algorithm. In contrast to established methods for sparse training the proposed…
▽ More
We propose a learning framework based on stochastic Bregman iterations, also known as mirror descent, to train sparse neural networks with an inverse scale space approach. We derive a baseline algorithm called LinBreg, an accelerated version using momentum, and AdaBreg, which is a Bregmanized generalization of the Adam algorithm. In contrast to established methods for sparse training the proposed family of algorithms constitutes a regrowth strategy for neural networks that is solely optimization-based without additional heuristics. Our Bregman learning framework starts the training with very few initial parameters, successively adding only significant ones to obtain a sparse and expressive network. The proposed approach is extremely easy and efficient, yet supported by the rich mathematical theory of inverse scale space methods. We derive a statistically profound sparse parameter initialization strategy and provide a rigorous stochastic convergence analysis of the loss decay and additional convergence proofs in the convex regime. Using only 3.4% of the parameters of ResNet-18 we achieve 90.2% test accuracy on CIFAR-10, compared to 93.6% using the dense network. Our algorithm also unveils an autoencoder architecture for a denoising task. The proposed framework also has a huge potential for integrating sparse backpropagation and resource-friendly training.
△ Less
Submitted 17 February, 2022; v1 submitted 10 May, 2021;
originally announced May 2021.
-
Complete Deterministic Dynamics and Spectral Decomposition of the Linear Ensemble Kalman Inversion
Authors:
Leon Bungert,
Philipp Wacker
Abstract:
The ensemble Kalman inversion (EKI) for the solution of Bayesian inverse problems of type $y = A u +\varepsilon$, with $u$ being an unknown parameter, $y$ a given datum, and $\varepsilon$ measurement noise, is a powerful tool usually derived from a sequential Monte Carlo point of view. It describes the dynamics of an ensemble of particles $\{u^j(t)\}_{j=1}^J$, whose initial empirical measure is sa…
▽ More
The ensemble Kalman inversion (EKI) for the solution of Bayesian inverse problems of type $y = A u +\varepsilon$, with $u$ being an unknown parameter, $y$ a given datum, and $\varepsilon$ measurement noise, is a powerful tool usually derived from a sequential Monte Carlo point of view. It describes the dynamics of an ensemble of particles $\{u^j(t)\}_{j=1}^J$, whose initial empirical measure is sampled from the prior, evolving over an artificial time $t$ towards an approximate solution of the inverse problem, with $t=1$ emulating the posterior, and $t\to\infty$ corresponding to the under-regularized minimum-norm solution of the inverse problem. Using spectral techniques, we provide a complete description of the deterministic dynamics of EKI and its asymptotic behavior in parameter space. In particular, we analyze the dynamics of naive EKI and mean-field EKI with a special focus on their time asymptotic behavior. Furthermore, we show that -- even in the deterministic case -- residuals in parameter space do not decrease monotonously in the Euclidean norm and suggest a problem-adapted norm, where monotonicity can be proved. Finally, we derive a system of ordinary differential equations governing the spectrum and eigenvectors of the covariance matrix. While the analysis is aimed at the EKI, we believe that it can be applied to understand more general particle-based dynamical systems.
△ Less
Submitted 30 October, 2022; v1 submitted 27 April, 2021;
originally announced April 2021.
-
CLIP: Cheap Lipschitz Training of Neural Networks
Authors:
Leon Bungert,
René Raab,
Tim Roith,
Leo Schwinn,
Daniel Tenbrinck
Abstract:
Despite the large success of deep neural networks (DNN) in recent years, most neural networks still lack mathematical guarantees in terms of stability. For instance, DNNs are vulnerable to small or even imperceptible input perturbations, so called adversarial examples, that can cause false predictions. This instability can have severe consequences in applications which influence the health and saf…
▽ More
Despite the large success of deep neural networks (DNN) in recent years, most neural networks still lack mathematical guarantees in terms of stability. For instance, DNNs are vulnerable to small or even imperceptible input perturbations, so called adversarial examples, that can cause false predictions. This instability can have severe consequences in applications which influence the health and safety of humans, e.g., biomedical imaging or autonomous driving. While bounding the Lipschitz constant of a neural network improves stability, most methods rely on restricting the Lipschitz constants of each layer which gives a poor bound for the actual Lipschitz constant.
In this paper we investigate a variational regularization method named CLIP for controlling the Lipschitz constant of a neural network, which can easily be integrated into the training procedure. We mathematically analyze the proposed model, in particular discussing the impact of the chosen regularization parameter on the output of the network. Finally, we numerically evaluate our method on both a nonlinear regression problem and the MNIST and Fashion-MNIST classification databases, and compare our results with a weight regularization approach.
△ Less
Submitted 31 October, 2022; v1 submitted 23 March, 2021;
originally announced March 2021.
-
Identifying Untrustworthy Predictions in Neural Networks by Geometric Gradient Analysis
Authors:
Leo Schwinn,
An Nguyen,
René Raab,
Leon Bungert,
Daniel Tenbrinck,
Dario Zanca,
Martin Burger,
Bjoern Eskofier
Abstract:
The susceptibility of deep neural networks to untrustworthy predictions, including out-of-distribution (OOD) data and adversarial examples, still prevent their widespread use in safety-critical applications. Most existing methods either require a re-training of a given model to achieve robust identification of adversarial attacks or are limited to out-of-distribution sample detection only. In this…
▽ More
The susceptibility of deep neural networks to untrustworthy predictions, including out-of-distribution (OOD) data and adversarial examples, still prevent their widespread use in safety-critical applications. Most existing methods either require a re-training of a given model to achieve robust identification of adversarial attacks or are limited to out-of-distribution sample detection only. In this work, we propose a geometric gradient analysis (GGA) to improve the identification of untrustworthy predictions without retraining of a given model. GGA analyzes the geometry of the loss landscape of neural networks based on the saliency maps of their respective input. To motivate the proposed approach, we provide theoretical connections between gradients' geometrical properties and local minima of the loss function. Furthermore, we demonstrate that the proposed method outperforms prior approaches in detecting OOD data and adversarial attacks, including state-of-the-art and adaptive attacks.
△ Less
Submitted 24 February, 2021;
originally announced February 2021.
-
Continuum Limit of Lipschitz Learning on Graphs
Authors:
Tim Roith,
Leon Bungert
Abstract:
Tackling semi-supervised learning problems with graph-based methods has become a trend in recent years since graphs can represent all kinds of data and provide a suitable framework for studying continuum limits, e.g., of differential operators. A popular strategy here is $p$-Laplacian learning, which poses a smoothness condition on the sought inference function on the set of unlabeled data. For…
▽ More
Tackling semi-supervised learning problems with graph-based methods has become a trend in recent years since graphs can represent all kinds of data and provide a suitable framework for studying continuum limits, e.g., of differential operators. A popular strategy here is $p$-Laplacian learning, which poses a smoothness condition on the sought inference function on the set of unlabeled data. For $p<\infty$ continuum limits of this approach were studied using tools from $Γ$-convergence. For the case $p=\infty$, which is referred to as Lipschitz learning, continuum limits of the related infinity-Laplacian equation were studied using the concept of viscosity solutions.
In this work, we prove continuum limits of Lipschitz learning using $Γ$-convergence. In particular, we define a sequence of functionals which approximate the largest local Lipschitz constant of a graph function and prove $Γ$-convergence in the $L^\infty$-topology to the supremum norm of the gradient as the graph becomes denser. Furthermore, we show compactness of the functionals which implies convergence of minimizers. In our analysis we allow a varying set of labeled data which converges to a general closed set in the Hausdorff distance. We apply our results to nonlinear ground states, i.e., minimizers with constrained $L^p$-norm, and, as a by-product, prove convergence of graph distance functions to geodesic distance functions.
△ Less
Submitted 29 November, 2021; v1 submitted 7 December, 2020;
originally announced December 2020.
-
The lion in the attic -- A resolution of the Borel--Kolmogorov paradox
Authors:
Leon Bungert,
Philipp Wacker
Abstract:
The Borel--Kolmogorov paradox of conditioning with respect to events of prior probability zero has fascinated students and researchers since its discovery more than 100 years ago. Classical conditioning is only valid with respect to events of positive probability. If we ignore this constraint and condition on such sets, for example events of type $\{Y=y\}$ for a continuously distributed random var…
▽ More
The Borel--Kolmogorov paradox of conditioning with respect to events of prior probability zero has fascinated students and researchers since its discovery more than 100 years ago. Classical conditioning is only valid with respect to events of positive probability. If we ignore this constraint and condition on such sets, for example events of type $\{Y=y\}$ for a continuously distributed random variable $Y$, almost any probability measure can be chosen as the conditional measure on such sets. There have been numerous descriptions and explanations of the paradox' appearance in the setting of conditioning on a subset of probability zero. However, most treatments don't supply explicit instructions on how to avoid it. We propose to close this gap by defining a version of conditional measure which utilizes the Hausdorff measure. This makes the choice canonical in the sense that it only depends on the geometry of the space, thus removing any ambiguity. We describe the set of possible measures arising in the context of the Borel--Kolmogorov paradox and classify those coinciding with the canonical measure. The objective of this manuscript is to provide a manual for singular conditional probability: We give an explicit explanation in which settings ambiguity arises (and where not) and how to get rid of this ambiguity once and for all by a canonical choice.
△ Less
Submitted 1 April, 2022; v1 submitted 10 September, 2020;
originally announced September 2020.
-
Variational regularisation for inverse problems with imperfect forward operators and general noise models
Authors:
Leon Bungert,
Martin Burger,
Yury Korolev,
Carola-Bibiane Schoenlieb
Abstract:
We study variational regularisation methods for inverse problems with imperfect forward operators whose errors can be modelled by order intervals in a partial order of a Banach lattice. We carry out analysis with respect to existence and convex duality for general data fidelity terms and regularisation functionals. Both for a-priori and a-posteriori parameter choice rules, we obtain convergence ra…
▽ More
We study variational regularisation methods for inverse problems with imperfect forward operators whose errors can be modelled by order intervals in a partial order of a Banach lattice. We carry out analysis with respect to existence and convex duality for general data fidelity terms and regularisation functionals. Both for a-priori and a-posteriori parameter choice rules, we obtain convergence rates of the regularized solutions in terms of Bregman distances. Our results apply to fidelity terms such as Wasserstein distances, f-divergences, norms, as well as sums and infimal convolutions of those.
△ Less
Submitted 23 October, 2020; v1 submitted 28 May, 2020;
originally announced May 2020.
-
The Infinity Laplacian eigenvalue problem: reformulation and a numerical scheme
Authors:
Farid Bozorgnia,
Leon Bungert,
Daniel Tenbrinck
Abstract:
In this work, we present an alternative formulation of the higher eigenvalue problem associated to the infinity Laplacian, which opens the door for numerical approximation of eigenfunctions. A rigorous analysis is performed to show the equivalence of the new formulation to the traditional one. Subsequently, we present consistent monotone schemes to approximate infinity ground states and higher eig…
▽ More
In this work, we present an alternative formulation of the higher eigenvalue problem associated to the infinity Laplacian, which opens the door for numerical approximation of eigenfunctions. A rigorous analysis is performed to show the equivalence of the new formulation to the traditional one. Subsequently, we present consistent monotone schemes to approximate infinity ground states and higher eigenfunctions on grids. We prove that our method converges (up to a subsequence) to a viscosity solution of the eigenvalue problem, and perform numerical experiments which investigate theoretical conjectures and compute eigenfunctions on a variety of different domains.
△ Less
Submitted 28 November, 2023; v1 submitted 17 April, 2020;
originally announced April 2020.
-
Robust Image Reconstruction with Misaligned Structural Information
Authors:
Leon Bungert,
Matthias J. Ehrhardt
Abstract:
Multi-modality (or multi-channel) imaging is becoming increasingly important and more widely available, e.g. hyperspectral imaging in remote sensing, spectral CT in material sciences as well as multi-contrast MRI and PET-MR in medicine. Research in the last decades resulted in a plethora of mathematical methods to combine data from several modalities. State-of-the-art methods, often formulated as…
▽ More
Multi-modality (or multi-channel) imaging is becoming increasingly important and more widely available, e.g. hyperspectral imaging in remote sensing, spectral CT in material sciences as well as multi-contrast MRI and PET-MR in medicine. Research in the last decades resulted in a plethora of mathematical methods to combine data from several modalities. State-of-the-art methods, often formulated as variational regularization, have shown to significantly improve image reconstruction both quantitatively and qualitatively. Almost all of these models rely on the assumption that the modalities are perfectly registered, which is not the case in most real world applications. We propose a variational framework which jointly performs reconstruction and registration, thereby overcoming this hurdle. Our approach is the first to achieve this for different modalities and outranks established approaches in terms of accuracy of both reconstruction and registration. Numerical results on simulated and real data show the potential of the proposed strategy for various applications in multi-contrast MRI, PET-MR, and hyperspectral imaging: typical misalignments between modalities such as rotations, translations, zooms can be effectively corrected during the reconstruction process. Therefore the proposed framework allows the robust exploitation of shared information across multiple modalities under real conditions.
△ Less
Submitted 24 December, 2020; v1 submitted 1 April, 2020;
originally announced April 2020.
-
Nonlinear Power Method for Computing Eigenvectors of Proximal Operators and Neural Networks
Authors:
Leon Bungert,
Ester Hait-Fraenkel,
Nicolas Papadakis,
Guy Gilboa
Abstract:
Neural networks have revolutionized the field of data science, yielding remarkable solutions in a data-driven manner. For instance, in the field of mathematical imaging, they have surpassed traditional methods based on convex regularization. However, a fundamental theory supporting the practical applications is still in the early stages of development. We take a fresh look at neural networks and e…
▽ More
Neural networks have revolutionized the field of data science, yielding remarkable solutions in a data-driven manner. For instance, in the field of mathematical imaging, they have surpassed traditional methods based on convex regularization. However, a fundamental theory supporting the practical applications is still in the early stages of development. We take a fresh look at neural networks and examine them via nonlinear eigenvalue analysis. The field of nonlinear spectral theory is still emerging, providing insights about nonlinear operators and systems. In this paper we view a neural network as a complex nonlinear operator and attempt to find its nonlinear eigenvectors. We first discuss the existence of such eigenvectors and analyze the kernel of ReLU networks. Then we study a nonlinear power method for generic nonlinear operators. For proximal operators associated to absolutely one-homogeneous convex regularization functionals, we can prove convergence of the method to an eigenvector of the proximal operator. This motivates us to apply a nonlinear method to networks which are trained to act similarly as a proximal operator. In order to take the non-homogeneity of neural networks into account we define a modified version of the power method.
We perform extensive experiments for different proximal operators and on various shallow and deep neural networks designed for image denoising. Proximal eigenvectors will be used for geometric analysis of graphs, as clustering or the computation of distance functions. For simple neural nets, we observe the influence of training data on the eigenvectors. For state-of-the-art denoising networks, we show that eigenvectors can be interpreted as (un)stable modes of the network, when contaminated with noise or other degradations.
△ Less
Submitted 19 April, 2021; v1 submitted 10 March, 2020;
originally announced March 2020.
-
Structural analysis of an $L$-infinity variational problem and relations to distance functions
Authors:
Leon Bungert,
Yury Korolev,
Martin Burger
Abstract:
In this work we analyse the functional ${\cal J}(u)=\|\nabla u\|_\infty$ defined on Lipschitz functions with homogeneous Dirichlet boundary conditions. Our analysis is performed directly on the functional without the need to approximate with smooth $p$-norms. We prove that its ground states coincide with multiples of the distance function to the boundary of the domain. Furthermore, we compute the…
▽ More
In this work we analyse the functional ${\cal J}(u)=\|\nabla u\|_\infty$ defined on Lipschitz functions with homogeneous Dirichlet boundary conditions. Our analysis is performed directly on the functional without the need to approximate with smooth $p$-norms. We prove that its ground states coincide with multiples of the distance function to the boundary of the domain. Furthermore, we compute the $L^2$-subdifferential of ${\cal J}$ and characterize the distance function as unique non-negative eigenfunction of the subdifferential operator. We also study properties of general eigenfunctions, in particular their nodal sets. Furthermore, we prove that the distance function can be computed as asymptotic profile of the gradient flow of ${\cal J}$ and construct analytic solutions of fast marching type. In addition, we give a geometric characterization of the extreme points of the unit ball of ${\cal J}$.
Finally, we transfer many of these results to a discrete version of the functional defined on a finite weighted graph. Here, we analyze properties of distance functions on graphs and their gradients. The main difference between the continuum and discrete setting is that the distance function is not the unique non-negative eigenfunction on a graph.
△ Less
Submitted 13 July, 2020; v1 submitted 21 January, 2020;
originally announced January 2020.
-
Asymptotic Profiles of Nonlinear Homogeneous Evolution Equations of Gradient Flow Type
Authors:
Leon Bungert,
Martin Burger
Abstract:
This work is concerned with the gradient flow of absolutely $p$-homogeneous convex functionals on a Hilbert space, which we show to exhibit finite ($p<2$) or infinite extinction time ($p \geq 2$). We give upper bounds for the finite extinction time and establish convergence rates of the flow. Moreover, we study next order asymptotics and prove that asymptotic profiles of the solution are eigenfunc…
▽ More
This work is concerned with the gradient flow of absolutely $p$-homogeneous convex functionals on a Hilbert space, which we show to exhibit finite ($p<2$) or infinite extinction time ($p \geq 2$). We give upper bounds for the finite extinction time and establish convergence rates of the flow. Moreover, we study next order asymptotics and prove that asymptotic profiles of the solution are eigenfunctions of the subdifferential operator of the functional. To this end, we compare with solutions of an ordinary differential equation which describes the evolution of eigenfunction under the flow. Our work applies, for instance, to local and nonlocal versions of PDEs like $p$-Laplacian evolution equations, the porous medium equation, and fast diffusion equations, herewith generalizing many results from the literature to an abstract setting.
We also demonstrate how our theory extends to general homogeneous evolution equations which are not necessarily a gradient flow. Here we discover an interesting integrability condition which characterizes whether or not asymptotic profiles are eigenfunctions.
△ Less
Submitted 10 June, 2020; v1 submitted 24 June, 2019;
originally announced June 2019.
-
Computing Nonlinear Eigenfunctions via Gradient Flow Extinction
Authors:
Leon Bungert,
Martin Burger,
Daniel Tenbrinck
Abstract:
In this work we investigate the computation of nonlinear eigenfunctions via the extinction profiles of gradient flows. We analyze a scheme that recursively subtracts such eigenfunctions from given data and show that this procedure yields a decomposition of the data into eigenfunctions in some cases as the 1-dimensional total variation, for instance. We discuss results of numerical experiments in w…
▽ More
In this work we investigate the computation of nonlinear eigenfunctions via the extinction profiles of gradient flows. We analyze a scheme that recursively subtracts such eigenfunctions from given data and show that this procedure yields a decomposition of the data into eigenfunctions in some cases as the 1-dimensional total variation, for instance. We discuss results of numerical experiments in which we use extinction profiles and the gradient flow for the task of spectral graph clustering as used, e.g., in machine learning applications.
△ Less
Submitted 27 February, 2019;
originally announced February 2019.
-
Nonlinear Spectral Decompositions by Gradient Flows of One-Homogeneous Functionals
Authors:
Leon Bungert,
Martin Burger,
Antonin Chambolle,
Matteo Novaga
Abstract:
This paper establishes a theory of nonlinear spectral decompositions by considering the eigenvalue problem related to an absolutely one-homogeneous functional in an infinite-dimensional Hilbert space. This approach is both motivated by works for the total variation, where interesting results on the eigenvalue problem and the relation to the total variation flow have been proven previously, and by…
▽ More
This paper establishes a theory of nonlinear spectral decompositions by considering the eigenvalue problem related to an absolutely one-homogeneous functional in an infinite-dimensional Hilbert space. This approach is both motivated by works for the total variation, where interesting results on the eigenvalue problem and the relation to the total variation flow have been proven previously, and by recent results on finite-dimensional polyhedral semi-norms, where gradient flows can yield spectral decompositions into eigenvectors.
We provide a geometric characterization of eigenvectors via a dual unit ball and prove them to be subgradients of minimal norm. This establishes the connection to gradient flows, whose time evolution is a decomposition of the initial condition into subgradients of minimal norm. If these are eigenvectors, this implies an interesting orthogonality relation and the equivalence of the gradient flow to a variational regularization method and an inverse scale space flow. Indeed we verify that all scenarios where these equivalences were known before by other arguments - such as one-dimensional total variation, multidimensional generalizations to vector fields, or certain polyhedral semi-norms - yield spectral decompositions, and we provide further examples. We also investigate extinction times and extinction profiles, which we characterize as eigenvectors in a very general setting, generalizing several results from literature.
△ Less
Submitted 19 September, 2021; v1 submitted 21 January, 2019;
originally announced January 2019.
-
Solution Paths of Variational Regularization Methods for Inverse Problems
Authors:
Leon Bungert,
Martin Burger
Abstract:
We consider a family of variational regularization functionals for a generic inverse problem, where the data fidelity and regularization term are given by powers of a Hilbert norm and an absolutely one-homogeneous functional, respectively, and the regularization parameter is interpreted as artificial time. We investigate the small and large time behavior of the associated solution paths and, in pa…
▽ More
We consider a family of variational regularization functionals for a generic inverse problem, where the data fidelity and regularization term are given by powers of a Hilbert norm and an absolutely one-homogeneous functional, respectively, and the regularization parameter is interpreted as artificial time. We investigate the small and large time behavior of the associated solution paths and, in particular, prove finite extinction time for a large class of functionals. Depending on the powers, we also show that the solution paths are of bounded variation or even Lipschitz continuous. In addition, it will turn out that the models are "almost" mutually equivalent in terms of the minimizers they admit. Finally, we apply our results to define and compare two different non-linear spectral representations of data and show that only one of it is able to decompose a linear combination of non-linear eigenfunctions into the individual eigenfunctions. For that purpose, we will also briefly address piecewise affine solution paths.
△ Less
Submitted 29 October, 2019; v1 submitted 6 August, 2018;
originally announced August 2018.
-
Blind Image Fusion for Hyperspectral Imaging with the Directional Total Variation
Authors:
Leon Bungert,
David A. Coomes,
Matthias J. Ehrhardt,
Jennifer Rasch,
Rafael Reisenhofer,
Carola-Bibiane Schönlieb
Abstract:
Hyperspectral imaging is a cutting-edge type of remote sensing used for mapping vegetation properties, rock minerals and other materials. A major drawback of hyperspectral imaging devices is their intrinsic low spatial resolution. In this paper, we propose a method for increasing the spatial resolution of a hyperspectral image by fusing it with an image of higher spatial resolution that was obtain…
▽ More
Hyperspectral imaging is a cutting-edge type of remote sensing used for mapping vegetation properties, rock minerals and other materials. A major drawback of hyperspectral imaging devices is their intrinsic low spatial resolution. In this paper, we propose a method for increasing the spatial resolution of a hyperspectral image by fusing it with an image of higher spatial resolution that was obtained with a different imaging modality. This is accomplished by solving a variational problem in which the regularization functional is the directional total variation. To accommodate for possible mis-registrations between the two images, we consider a non-convex blind super-resolution problem where both a fused image and the corresponding convolution kernel are estimated. Using this approach, our model can realign the given images if needed. Our experimental results indicate that the non-convexity is negligible in practice and that reliable solutions can be computed using a variety of different optimization algorithms. Numerical results on real remote sensing data from plant sciences and urban monitoring show the potential of the proposed method and suggests that it is robust with respect to the regularization parameters, mis-registration and the shape of the kernel.
△ Less
Submitted 9 April, 2018; v1 submitted 4 October, 2017;
originally announced October 2017.
