-
Representing Piecewise Linear Functions by Functions with Small Arity
Authors:
Christoph Koutschan,
Bernhard Moser,
Anton Ponomarchuk,
Josef Schicho
Abstract:
A piecewise linear function can be described in different forms: as an arbitrarily nested expression of $\min$- and $\max$-functions, as a difference of two convex piecewise linear functions, or as a linear combination of maxima of affine-linear functions. In this paper, we provide two main results: first, we show that for every piecewise linear function there exists a linear combination of…
▽ More
A piecewise linear function can be described in different forms: as an arbitrarily nested expression of $\min$- and $\max$-functions, as a difference of two convex piecewise linear functions, or as a linear combination of maxima of affine-linear functions. In this paper, we provide two main results: first, we show that for every piecewise linear function there exists a linear combination of $\max$-functions with at most $n+1$ arguments, and give an algorithm for its computation. Moreover, these arguments are contained in the finite set of affine-linear functions that coincide with the given function in some open set. Second, we prove that the piecewise linear function $\max(0, x_{1}, \ldots, x_{n})$ cannot be represented as a linear combination of maxima of less than $n+1$ affine-linear arguments. This was conjectured by Wang and Sun in 2005 in a paper on representations of piecewise linear functions as linear combination of maxima.
△ Less
Submitted 26 May, 2023;
originally announced May 2023.
-
Spiking Neural Networks in the Alexiewicz Topology: A New Perspective on Analysis and Error Bounds
Authors:
Bernhard A. Moser,
Michael Lunglmayr
Abstract:
In order to ease the analysis of error propagation in neuromorphic computing and to get a better understanding of spiking neural networks (SNN), we address the problem of mathematical analysis of SNNs as endomorphisms that map spike trains to spike trains. A central question is the adequate structure for a space of spike trains and its implication for the design of error measurements of SNNs inclu…
▽ More
In order to ease the analysis of error propagation in neuromorphic computing and to get a better understanding of spiking neural networks (SNN), we address the problem of mathematical analysis of SNNs as endomorphisms that map spike trains to spike trains. A central question is the adequate structure for a space of spike trains and its implication for the design of error measurements of SNNs including time delay, threshold deviations, and the design of the reinitialization mode of the leaky-integrate-and-fire (LIF) neuron model. First we identify the underlying topology by analyzing the closure of all sub-threshold signals of a LIF model. For zero leakage this approach yields the Alexiewicz topology, which we adopt to LIF neurons with arbitrary positive leakage. As a result LIF can be understood as spike train quantization in the corresponding norm. This way we obtain various error bounds and inequalities such as a quasi isometry relation between incoming and outgoing spike trains. Another result is a Lipschitz-style global upper bound for the error propagation and a related resonance-type phenomenon.
△ Less
Submitted 8 February, 2024; v1 submitted 9 May, 2023;
originally announced May 2023.
-
Addressing Parameter Choice Issues in Unsupervised Domain Adaptation by Aggregation
Authors:
Marius-Constantin Dinu,
Markus Holzleitner,
Maximilian Beck,
Hoan Duc Nguyen,
Andrea Huber,
Hamid Eghbal-zadeh,
Bernhard A. Moser,
Sergei Pereverzyev,
Sepp Hochreiter,
Werner Zellinger
Abstract:
We study the problem of choosing algorithm hyper-parameters in unsupervised domain adaptation, i.e., with labeled data in a source domain and unlabeled data in a target domain, drawn from a different input distribution. We follow the strategy to compute several models using different hyper-parameters, and, to subsequently compute a linear aggregation of the models. While several heuristics exist t…
▽ More
We study the problem of choosing algorithm hyper-parameters in unsupervised domain adaptation, i.e., with labeled data in a source domain and unlabeled data in a target domain, drawn from a different input distribution. We follow the strategy to compute several models using different hyper-parameters, and, to subsequently compute a linear aggregation of the models. While several heuristics exist that follow this strategy, methods are still missing that rely on thorough theories for bounding the target error. In this turn, we propose a method that extends weighted least squares to vector-valued functions, e.g., deep neural networks. We show that the target error of the proposed algorithm is asymptotically not worse than twice the error of the unknown optimal aggregation. We also perform a large scale empirical comparative study on several datasets, including text, images, electroencephalogram, body sensor signals and signals from mobile phones. Our method outperforms deep embedded validation (DEV) and importance weighted validation (IWV) on all datasets, setting a new state-of-the-art performance for solving parameter choice issues in unsupervised domain adaptation with theoretical error guarantees. We further study several competitive heuristics, all outperforming IWV and DEV on at least five datasets. However, our method outperforms each heuristic on at least five of seven datasets.
△ Less
Submitted 2 May, 2023;
originally announced May 2023.
-
Construction of adaptive exponential multi-operator splitting methods
Authors:
Othmar Koch,
Koray Acar,
Winfried Auzinger,
Daniel Hoffmann,
Friedrich Kupka,
Benedikt Moser
Abstract:
We construct splitting methods suitable for the solution of the equations of magnetohydrodynamics (MHD). Due to the physical significance of the involved operators, splittings into three or even four operators with positive coefficients are appropriate for a physically correct and efficient solution of the equations. To efficiently obtain an accurate solution approximation, adaptive choice of the…
▽ More
We construct splitting methods suitable for the solution of the equations of magnetohydrodynamics (MHD). Due to the physical significance of the involved operators, splittings into three or even four operators with positive coefficients are appropriate for a physically correct and efficient solution of the equations. To efficiently obtain an accurate solution approximation, adaptive choice of the time-steps is important particularly in the light of the unsmooth dynamics of the system. Thus, we construct new method coefficients in conjunction with associated error estimators by optimizing the leading local error term. As a proof of concept, we demonstrate that adaptive splitting faithfully reflects the solution behavior also in the presence of a shock-like behavior for the viscous Burgers equation, which serves as a simplified model problem displaying several features of the Navier-Stokes equation for incompressible flow.
△ Less
Submitted 11 March, 2025; v1 submitted 2 February, 2023;
originally announced February 2023.
-
Membership-Mappings for Data Representation Learning: Measure Theoretic Conceptualization
Authors:
Mohit Kumar,
Bernhard A. Moser,
Lukas Fischer,
Bernhard Freudenthaler
Abstract:
A fuzzy theoretic analytical approach was recently introduced that leads to efficient and robust models while addressing automatically the typical issues associated to parametric deep models. However, a formal conceptualization of the fuzzy theoretic analytical deep models is still not available. This paper introduces using measure theoretic basis the notion of \emph{membership-mapping} for repres…
▽ More
A fuzzy theoretic analytical approach was recently introduced that leads to efficient and robust models while addressing automatically the typical issues associated to parametric deep models. However, a formal conceptualization of the fuzzy theoretic analytical deep models is still not available. This paper introduces using measure theoretic basis the notion of \emph{membership-mapping} for representing data points through attribute values (motivated by fuzzy theory). A property of the membership-mapping, that can be exploited for data representation learning, is of providing an interpolation on the given data points in the data space. An analytical approach to the variational learning of a membership-mappings based data representation model is considered.
△ Less
Submitted 10 June, 2022; v1 submitted 14 April, 2021;
originally announced April 2021.
-
A Novel Fibonacci Pattern in Pascal's Triangle
Authors:
Bernhard Moser
Abstract:
The Fibonacci sequence is obtained as weighted sum along the rows in the Pascal triangle by choosing a periodic up-and-down pattern of weights from the set $\{-1,-\frac{1}{2},0, \frac{1}{2}, 1\}$. A graphical illustration of this identity shows a novel "`beautiful"' Fibonacci pattern.
The Fibonacci sequence is obtained as weighted sum along the rows in the Pascal triangle by choosing a periodic up-and-down pattern of weights from the set $\{-1,-\frac{1}{2},0, \frac{1}{2}, 1\}$. A graphical illustration of this identity shows a novel "`beautiful"' Fibonacci pattern.
△ Less
Submitted 5 November, 2018;
originally announced November 2018.
-
On the Discrepancy Normed Space of Event Sequences for Threshold-based Sampling
Authors:
Bernhard A. Moser
Abstract:
Recalling recent results on the characterization of threshold-based sampling as quasi-isometric mapping, mathematical implications on the metric and topological structure of the space of event sequences are derived. In this context, the space of event sequences is extended to a normed space equipped with Hermann Weyl's discrepancy measure. Sequences of finite discrepancy norm are characterized by…
▽ More
Recalling recent results on the characterization of threshold-based sampling as quasi-isometric mapping, mathematical implications on the metric and topological structure of the space of event sequences are derived. In this context, the space of event sequences is extended to a normed space equipped with Hermann Weyl's discrepancy measure. Sequences of finite discrepancy norm are characterized by a Jordan decomposition property. Its dual norm turns out to be the norm of total variation. As a by-product a measure for the lack of monotonicity of sequences is obtained. A further result refers to an inequality between the discrepancy norm and total variation which resembles Heisenberg's uncertainty relation.
△ Less
Submitted 16 June, 2018;
originally announced June 2018.
