-
A precise bare simulation approach to the minimization of some distances. II. Further Foundations
Authors:
Michel Broniatowski,
Wolfgang Stummer
Abstract:
The constrained minimization (respectively maximization) of directed distances and of related generalized entropies is a fundamental task in information theory as well as in the adjacent fields of statistics, machine learning, artificial intelligence, signal processing and pattern recognition. In our previous paper "A precise bare simulation approach to the minimization of some distances. I. Found…
▽ More
The constrained minimization (respectively maximization) of directed distances and of related generalized entropies is a fundamental task in information theory as well as in the adjacent fields of statistics, machine learning, artificial intelligence, signal processing and pattern recognition. In our previous paper "A precise bare simulation approach to the minimization of some distances. I. Foundations", we obtained such kind of constrained optima by a new dimension-free precise bare (pure) simulation method, provided basically that (i) the underlying directed distance is of f-divergence type, and that (ii) this can be connected to a light-tailed probability distribution in a certain manner. In the present paper, we extend this approach such that constrained optimization problems of a very huge amount of directed distances and generalized entropies -- and beyond -- can be tackled by a newly developed dimension-free extended bare simulation method, for obtaining both optima as well as optimizers. Almost no assumptions (like convexity) on the set of constraints are needed, within our discrete setup of arbitrary dimension, and our method is precise (i.e., converges in the limit). For instance, we cover constrained optimizations of arbitrary f-divergences, Bregman distances, scaled Bregman distances and weighted Euclidean distances. The potential for wide-spread applicability is indicated, too; in particular, we deliver many recent references for uses of the involved distances/divergences in various different research fields (which may also serve as an interdisciplinary interface).
△ Less
Submitted 13 February, 2024;
originally announced February 2024.
-
A Unifying Framework for Some Directed Distances in Statistics
Authors:
Michel Broniatowski,
Wolfgang Stummer
Abstract:
Density-based directed distances -- particularly known as divergences -- between probability distributions are widely used in statistics as well as in the adjacent research fields of information theory, artificial intelligence and machine learning. Prominent examples are the Kullback-Leibler information distance (relative entropy) which e.g. is closely connected to the omnipresent maximum likeliho…
▽ More
Density-based directed distances -- particularly known as divergences -- between probability distributions are widely used in statistics as well as in the adjacent research fields of information theory, artificial intelligence and machine learning. Prominent examples are the Kullback-Leibler information distance (relative entropy) which e.g. is closely connected to the omnipresent maximum likelihood estimation method, and Pearson's chisquare-distance which e.g. is used for the celebrated chisquare goodness-of-fit test. Another line of statistical inference is built upon distribution-function-based divergences such as e.g. the prominent (weighted versions of) Cramer-von Mises test statistics respectively Anderson-Darling test statistics which are frequently applied for goodness-of-fit investigations; some more recent methods deal with (other kinds of) cumulative paired divergences and closely related concepts. In this paper, we provide a general framework which covers in particular both the above-mentioned density-based and distribution-function-based divergence approaches; the dissimilarity of quantiles respectively of other statistical functionals will be included as well. From this framework, we structurally extract numerous classical and also state-of-the-art (including new) procedures. Furthermore, we deduce new concepts of dependence between random variables, as alternatives to the celebrated mutual information. Some variational representations are discussed, too.
△ Less
Submitted 1 March, 2022;
originally announced March 2022.
-
A precise bare simulation approach to the minimization of some distances. Foundations
Authors:
Michel Broniatowski,
Wolfgang Stummer
Abstract:
In information theory -- as well as in the adjacent fields of statistics, machine learning, artificial intelligence, signal processing and pattern recognition -- many flexibilizations of the omnipresent Kullback-Leibler information distance (relative entropy) and of the closely related Shannon entropy have become frequently used tools. To tackle corresponding constrained minimization (respectively…
▽ More
In information theory -- as well as in the adjacent fields of statistics, machine learning, artificial intelligence, signal processing and pattern recognition -- many flexibilizations of the omnipresent Kullback-Leibler information distance (relative entropy) and of the closely related Shannon entropy have become frequently used tools. To tackle corresponding constrained minimization (respectively maximization) problems by a newly developed dimension-free bare (pure) simulation method, is the main goal of this paper. Almost no assumptions (like convexity) on the set of constraints are needed, within our discrete setup of arbitrary dimension, and our method is precise (i.e., converges in the limit). As a side effect, we also derive an innovative way of constructing new useful distances/divergences. To illustrate the core of our approach, we present numerous solved cases. The potential for widespread applicability is indicated, too; in particular, we deliver many recent references for uses of the involved distances/divergences and entropies in various different research fields (which may also serve as an interdisciplinary interface).
△ Less
Submitted 15 November, 2022; v1 submitted 4 July, 2021;
originally announced July 2021.
-
Continuous indetermination and average likelihood minimization
Authors:
Pierre Bertrand,
Michel Broniatowski,
Jean-François Marcotorchino
Abstract:
The authors transpose a discrete notion of indetermination coupling in the case of continuous probabilities. They show that this coupling, expressed on densities, cannot be captured by a specific copula which acts on cumulative distribution functions without a high dependence on the margins. Furthermore, they define a notion of average likelihood which extends the discrete notion of couple matchin…
▽ More
The authors transpose a discrete notion of indetermination coupling in the case of continuous probabilities. They show that this coupling, expressed on densities, cannot be captured by a specific copula which acts on cumulative distribution functions without a high dependence on the margins. Furthermore, they define a notion of average likelihood which extends the discrete notion of couple matchings and demonstrate it is minimal under indetermination. Eventually, they leverage this property to build up a statistical test to distinguish indetermination and estimate its efficiency using the Bahadur's slope.
△ Less
Submitted 4 May, 2021;
originally announced May 2021.
-
A constructive method to minimize couple matchings
Authors:
Pierre Bertrand,
Michel Broniatowski,
Jean-François Marcotorchino
Abstract:
This paper provides constructive procedures for the indeterminacy coupling between two marginal distributions, an alternative to independence coupling. It also introduces a drawing under indeterminacy into a mixture of three independent couplings. Leveraging on this decomposition it states that indeterminacy optimally reduces couple matchings, minimizing the expected number of equal couples drawn…
▽ More
This paper provides constructive procedures for the indeterminacy coupling between two marginal distributions, an alternative to independence coupling. It also introduces a drawing under indeterminacy into a mixture of three independent couplings. Leveraging on this decomposition it states that indeterminacy optimally reduces couple matchings, minimizing the expected number of equal couples drawn in a row. Besides it is seen that the Janson Vegelius coefficient is nothing but a deviation to indeterminacy and it is shown that it tends to 0 when the number of modalities increases.
△ Less
Submitted 14 February, 2023; v1 submitted 29 December, 2020;
originally announced December 2020.
-
Independence versus Indetermination: basis of two canonical clustering criteria
Authors:
Pierre Bertrand,
Michel Broniatowski,
Jean-François Marcotorchino
Abstract:
This paper aims at comparing two coupling approaches as basic layers for building clustering criteria, suited for modularizing and clustering very large networks. We briefly use "optimal transport theory" as a starting point, and a way as well, to derive two canonical couplings: "statistical independence" and "logical indetermination". A symmetric list of properties is provided and notably the so…
▽ More
This paper aims at comparing two coupling approaches as basic layers for building clustering criteria, suited for modularizing and clustering very large networks. We briefly use "optimal transport theory" as a starting point, and a way as well, to derive two canonical couplings: "statistical independence" and "logical indetermination". A symmetric list of properties is provided and notably the so called "Monge's properties", applied to contingency matrices, and justifying the $\otimes$ versus $\oplus$ notation. A study is proposed, highlighting "logical indetermination", because it is, by far, lesser known. Eventually we estimate the average difference between both couplings as the key explanation of their usually close results in network clustering.
△ Less
Submitted 18 March, 2021; v1 submitted 17 July, 2020;
originally announced July 2020.
-
A recursive algorithm for a pipeline maintenance scheduling problem
Authors:
Assia Boumahdaf,
Michel Broniatowski
Abstract:
This paper deals with the problem of preventive maintenance (PM) scheduling of pipelines subject to external corrosion defects. The preventive maintenance strategy involves an inspection step at some epoch, together with a repair schedule. This paper proposes to determine the repair schedule as well as an inspection time minimizing the maintenance cost. This problem is formulated as a binary integ…
▽ More
This paper deals with the problem of preventive maintenance (PM) scheduling of pipelines subject to external corrosion defects. The preventive maintenance strategy involves an inspection step at some epoch, together with a repair schedule. This paper proposes to determine the repair schedule as well as an inspection time minimizing the maintenance cost. This problem is formulated as a binary integer non-linear programming model and we approach it under a decision support framework. We derive a polynomial-time algorithm that computes the optimum PM schedule and suggests different PM strategies in order to assist practitioners in making decision.
△ Less
Submitted 4 October, 2016;
originally announced October 2016.
