Showing 1–4 of 4 results for author: Kofnov, A

Exact Upper and Lower Bounds for the Output Distribution of Neural Networks with Random Inputs

Authors: Andrey Kofnov, Daniel Kapla, Ezio Bartocci, Efstathia Bura

Abstract: We derive exact upper and lower bounds for the cumulative distribution function (cdf) of the output of a neural network (NN) over its entire support subject to noisy (stochastic) inputs. The upper and lower bounds converge to the true cdf over its domain as the resolution increases. Our method applies to any feedforward NN using continuous monotonic piecewise twice continuously differentiable acti… ▽ More We derive exact upper and lower bounds for the cumulative distribution function (cdf) of the output of a neural network (NN) over its entire support subject to noisy (stochastic) inputs. The upper and lower bounds converge to the true cdf over its domain as the resolution increases. Our method applies to any feedforward NN using continuous monotonic piecewise twice continuously differentiable activation functions (e.g., ReLU, tanh and softmax) and convolutional NNs, which were beyond the scope of competing approaches. The novelty and instrumental tool of our approach is to bound general NNs with ReLU NNs. The ReLU NN-based bounds are then used to derive the upper and lower bounds of the cdf of the NN output. Experiments demonstrate that our method delivers guaranteed bounds of the predictive output distribution over its support, thus providing exact error guarantees, in contrast to competing approaches. △ Less

Submitted 10 June, 2025; v1 submitted 17 February, 2025; originally announced February 2025.

Comments: Forthcoming at the 42nd International Conference on Machine Learning (ICML 2025); 9+16 pages

MSC Class: 62E15 (Primary) 46N30; 62H10 (Secondary) ACM Class: G.3; I.5.1

Exact and Approximate Moment Derivation for Probabilistic Loops With Non-Polynomial Assignments

Authors: Andrey Kofnov, Marcel Moosbrugger, Miroslav Stankovič, Ezio Bartocci, Efstathia Bura

Abstract: Many stochastic continuous-state dynamical systems can be modeled as probabilistic programs with nonlinear non-polynomial updates in non-nested loops. We present two methods, one approximate and one exact, to automatically compute, without sampling, moment-based invariants for such probabilistic programs as closed-form solutions parameterized by the loop iteration. The exact method applies to prob… ▽ More Many stochastic continuous-state dynamical systems can be modeled as probabilistic programs with nonlinear non-polynomial updates in non-nested loops. We present two methods, one approximate and one exact, to automatically compute, without sampling, moment-based invariants for such probabilistic programs as closed-form solutions parameterized by the loop iteration. The exact method applies to probabilistic programs with trigonometric and exponential updates and is embedded in the Polar tool. The approximate method for moment computation applies to any nonlinear random function as it exploits the theory of polynomial chaos expansion to approximate non-polynomial updates as the sum of orthogonal polynomials. This translates the dynamical system to a non-nested loop with polynomial updates, and thus renders it conformable with the Polar tool that computes the moments of any order of the state variables. We evaluate our methods on an extensive number of examples ranging from modeling monetary policy to several physical motion systems in uncertain environments. The experimental results demonstrate the advantages of our approach with respect to the current state-of-the-art. △ Less

Submitted 25 January, 2024; v1 submitted 12 June, 2023; originally announced June 2023.

Comments: Published in ACM Transactions on Modeling and Computer Simulation (TOMACS). Extended version of the conference paper 'Moment-based Invariants for Probabilistic Loops with Non-polynomial Assignments' published at QEST 2022 (Best paper award, see also the preprint arxiv.org/abs/2205.02577). arXiv admin note: substantial text overlap with arXiv:2205.02577

MSC Class: 62G05; 62P30 ACM Class: G.3

Moment-based Density Elicitation with Applications in Probabilistic Loops

Authors: Andrey Kofnov, Ezio Bartocci, Efstathia Bura

Abstract: We propose the K-series estimation approach for the recovery of unknown univariate and multivariate distributions given knowledge of a finite number of their moments. Our method is directly applicable to the probabilistic analysis of systems that can be represented as probabilistic loops; i.e., algorithms that express and implement non-deterministic processes ranging from robotics to macroeconomic… ▽ More We propose the K-series estimation approach for the recovery of unknown univariate and multivariate distributions given knowledge of a finite number of their moments. Our method is directly applicable to the probabilistic analysis of systems that can be represented as probabilistic loops; i.e., algorithms that express and implement non-deterministic processes ranging from robotics to macroeconomics and biology to software and cyber-physical systems. K-series statically approximates the joint and marginal distributions of a vector of continuous random variables updated in a probabilistic non-nested loop with nonlinear assignments given a finite number of moments of the unknown density. Moreover, K-series automatically derives the distribution of the systems' random variables symbolically as a function of the loop iteration. K-series density estimates are accurate, easy and fast to compute. We demonstrate the feasibility and performance of our approach on multiple benchmark examples from the literature. △ Less

Submitted 11 April, 2025; v1 submitted 17 April, 2023; originally announced April 2023.

Comments: Accepted for publication in ACM Transactions on Probabilistic Machine Learning, 37 page

MSC Class: 62G07; 60E05 (Primary) 60B10 (Secondary) ACM Class: G.3; I.1.1

Moment-based Invariants for Probabilistic Loops with Non-polynomial Assignments

Authors: Andrey Kofnov, Marcel Moosbrugger, Miroslav Stankovič, Ezio Bartocci, Efstathia Bura

Abstract: We present a method to automatically approximate moment-based invariants of probabilistic programs with non-polynomial updates of continuous state variables to accommodate more complex dynamics. Our approach leverages polynomial chaos expansion to approximate non-linear functional updates as sums of orthogonal polynomials. We exploit this result to automatically estimate state-variable moments of… ▽ More We present a method to automatically approximate moment-based invariants of probabilistic programs with non-polynomial updates of continuous state variables to accommodate more complex dynamics. Our approach leverages polynomial chaos expansion to approximate non-linear functional updates as sums of orthogonal polynomials. We exploit this result to automatically estimate state-variable moments of all orders in Prob-solvable loops with non-polynomial updates. We showcase the accuracy of our estimation approach in several examples, such as the turning vehicle model and the Taylor rule in monetary policy. △ Less

Submitted 1 July, 2022; v1 submitted 5 May, 2022; originally announced May 2022.

Comments: 23 pages

MSC Class: 62G05 (Primary) 62P30 (Secondary) ACM Class: G.3