Showing 1–43 of 43 results for author: Madiman, M

Log-concavity in one-dimensional Coulomb gases and related ensembles

Authors: Jnaneshwar Baslingker, Manjunath Krishnapur, Mokshay Madiman

Abstract: We prove log-concavity of the lengths of the top rows of Young diagrams under Poissonized Plancherel measure. This is the first known positive result towards a conjecture of Chen that the length of the top row of a Young diagram under the Plancherel measure is log-concave. This is done by showing that the ordered elements of several discrete ensembles have log-concave distributions. In particular,… ▽ More We prove log-concavity of the lengths of the top rows of Young diagrams under Poissonized Plancherel measure. This is the first known positive result towards a conjecture of Chen that the length of the top row of a Young diagram under the Plancherel measure is log-concave. This is done by showing that the ordered elements of several discrete ensembles have log-concave distributions. In particular, we show the log-concavity of passage times in last passage percolation with geometric weights, using their connection to Meixner ensembles. In the continuous setting, distributions of the maximal elements of beta ensembles with convex potentials on the real line are shown to be log-concave. As a result, log-concavity of the $β$ versions of Tracy-Widom distributions follows; in fact, we also obtain log-concavity and positive association for the joint distribution of the $k$ smallest eigenvalues of the stochastic Airy operator. Our methods also show the log-concavity of finite dimensional distributions of the Airy-2 process and the Airy distribution. A log-concave distribution with full-dimensional support must have density, a fact that was apparently not known for some of these examples. △ Less

Submitted 19 December, 2024; originally announced December 2024.

Comments: 27 pages

The entropic doubling constant and robustness of Gaussian codebooks for additive-noise channels

Authors: Lampros Gavalakis, Ioannis Kontoyiannis, Mokshay Madiman

Abstract: Entropy comparison inequalities are obtained for the differential entropy $h(X+Y)$ of the sum of two independent random vectors $X,Y$, when one is replaced by a Gaussian. For identically distributed random vectors $X,Y$, these are closely related to bounds on the entropic doubling constant, which quantifies the entropy increase when adding an independent copy of a random vector to itself. Conseque… ▽ More Entropy comparison inequalities are obtained for the differential entropy $h(X+Y)$ of the sum of two independent random vectors $X,Y$, when one is replaced by a Gaussian. For identically distributed random vectors $X,Y$, these are closely related to bounds on the entropic doubling constant, which quantifies the entropy increase when adding an independent copy of a random vector to itself. Consequences of both large and small doubling are explored. For the former, lower bounds are deduced on the entropy increase when adding an independent Gaussian, while for the latter, a qualitative stability result for the entropy power inequality is obtained. In the more general case of non-identically distributed random vectors $X,Y$, a Gaussian comparison inequality with interesting implications for channel coding is established: For additive-noise channels with a power constraint, Gaussian codebooks come within a $\frac{\sf snr}{3{\sf snr}+2}$ factor of capacity. In the low-SNR regime this improves the half-a-bit additive bound of Zamir and Erez (2004). Analogous results are obtained for additive-noise multiple access channels, and for linear, additive-noise MIMO channels. △ Less

Submitted 23 July, 2024; v1 submitted 11 March, 2024; originally announced March 2024.

Comments: 23 pages, no figures. In this revised version we incorporate the reviewers' comments. In particular, a small error in the proof of Theorem 3 has been fixed

Weighted Brunn-Minkowski Theory II: Inequalities for Mixed Measures and Applications

Authors: Matthieu Fradelizi, Dylan Langharst, Mokshay Madiman, Artem Zvavitch

Abstract: In "Weighted Brunn-Minkowski Theory I", the prequel to this work, we discussed how recent developments on concavity of measures have laid the foundations of a nascent weighted Brunn-Minkowski theory. In particular, we defined the mixed measures of three convex bodies and obtained its integral representation. In this work, we obtain inequalities for mixed measures, such as a generalization of Fench… ▽ More In "Weighted Brunn-Minkowski Theory I", the prequel to this work, we discussed how recent developments on concavity of measures have laid the foundations of a nascent weighted Brunn-Minkowski theory. In particular, we defined the mixed measures of three convex bodies and obtained its integral representation. In this work, we obtain inequalities for mixed measures, such as a generalization of Fenchel's inequality; this provides a new, simpler proof of the classical volume case. Moreover, we show that mixed measures are connected to the study of log-submodularity and supermodularity of the measure of Minkowski sums of convex bodies. This elaborates on the recent investigations of these properties for the Lebesgue measure. We conclude by establishing that the only Radon measures that are supermodular over the class of compact, convex sets are multiples of the Lebesgue measure. Motivated by this result, we then discuss weaker forms of supermodularity by restricting the class of convex sets. △ Less

Submitted 5 October, 2024; v1 submitted 15 February, 2024; originally announced February 2024.

Comments: 35-37 pages

MSC Class: 52A20 and 52A21; Secondary: 46T12 2 52A20 and 52A21; Secondary: 46T12 2 52A20; 52A21; 46T12

A Closed-Form EVSI Expression for a Multinomial Data-Generating Process

Authors: Adam Fleischhacker, Pak-Wing Fok, Mokshay Madiman, Nan Wu

Abstract: This paper derives analytic expressions for the expected value of sample information (EVSI), the expected value of distribution information (EVDI), and the optimal sample size when data consists of independent draws from a bounded sequence of integers. Due to challenges of creating tractable EVSI expressions, most existing work valuing data does so in one of three ways: 1) analytically through clo… ▽ More This paper derives analytic expressions for the expected value of sample information (EVSI), the expected value of distribution information (EVDI), and the optimal sample size when data consists of independent draws from a bounded sequence of integers. Due to challenges of creating tractable EVSI expressions, most existing work valuing data does so in one of three ways: 1) analytically through closed-form expressions on the upper bound of the value of data, 2) calculating the expected value of data using numerical comparisons of decisions made using simulated data to optimal decisions where the underlying data distribution is known, or 3) using variance reduction as proxy for the uncertainty reduction that accompanies more data. For the very flexible case of modelling integer-valued observations using a multinomial data-generating process with Dirichlet prior, this paper develops expressions that 1) generalize existing beta-Binomial computations, 2) do not require prior knowledge of some underlying "true" distribution, and 3) can be computed prior to the collection of any sample data. △ Less

Submitted 1 December, 2022; originally announced January 2023.

Journal ref: Decision Analysis, Vol. 20, No. 1, pp. 73--84, March 2023

Weighted Brunn-Minkowski Theory I: On Weighted Surface Area Measures

Authors: Matthieu Fradelizi, Dylan Langharst, Mokshay Madiman, Artem Zvavitch

Abstract: The Brunn-Minkowski theory in convex geometry concerns, among other things, the volumes, mixed volumes, and surface area measures of convex bodies. We study generalizations of these concepts to Borel measures with density in $\mathbb{R}^n$-- in particular, the weighted versions of mixed volumes (the so-called mixed measures) when dealing with up to three distinct convex bodies. We then formulate a… ▽ More The Brunn-Minkowski theory in convex geometry concerns, among other things, the volumes, mixed volumes, and surface area measures of convex bodies. We study generalizations of these concepts to Borel measures with density in $\mathbb{R}^n$-- in particular, the weighted versions of mixed volumes (the so-called mixed measures) when dealing with up to three distinct convex bodies. We then formulate and analyze weighted versions of classical surface area measures, and obtain a new integral formula for the mixed measure of three bodies. As an application, we prove a Bézout-type inequality for rotational invariant log-concave measures, generalizing a result by Artstein-Avidan, Florentin and Ostrover. The results are new and interesting even for the special case of the standard Gaussian measure. △ Less

Submitted 3 May, 2023; v1 submitted 27 December, 2022; originally announced December 2022.

Comments: 30 pages, Keywords: Brunn-Minkowski theory, surface area, Gaussian measure, zonoids, mixed volumes, mixed measures. Previously titled "Weighted surface area measures and Brunn-Minkowski theory", the paper was split and improved upon

MSC Class: 52A20; 52A21; 46T12

Journal ref: Journal of Mathematical Analysis and Applications, Vol. 529, No. 2, January 2024

On the volume of the Minkowski sum of zonoids

Authors: Matthieu Fradelizi, Mokshay Madiman, Mathieu Meyer, Artem Zvavitch

Abstract: We explore some inequalities in convex geometry restricted to the class of zonoids. We show the equivalence, in the class of zonoids, between a local Alexandrov-Fenchel inequality, a local Loomis-Whitney inequality, the log-submodularity of volume, and the Dembo-Cover-Thomas conjecture on the monotonicity of the ratio of volume to the surface area. In addition to these equivalences, we confirm the… ▽ More We explore some inequalities in convex geometry restricted to the class of zonoids. We show the equivalence, in the class of zonoids, between a local Alexandrov-Fenchel inequality, a local Loomis-Whitney inequality, the log-submodularity of volume, and the Dembo-Cover-Thomas conjecture on the monotonicity of the ratio of volume to the surface area. In addition to these equivalences, we confirm these conjectures in ${\mathbb R}^3$ and we establish an improved inequality in ${\mathbb R^2}$. Along the way, we give a negative answer to a question of Adam Marcus regarding the roots of the Steiner polynomial of zonoids. We also investigate analogous questions in the $L_p$-Brunn-Minkowski theory, and in particular, we confirm all of the above conjectures in the case $p=2$, in any dimension. △ Less

Submitted 5 June, 2022; originally announced June 2022.

MSC Class: 52A40; 52A39; 52A20

Journal ref: Journal of Functional Analysis, Vol. 286, No. 3, February 2024

Sumset estimates in convex geometry

Authors: Matthieu Fradelizi, Mokshay Madiman, Artem Zvavitch

Abstract: Sumset estimates, which provide bounds on the cardinality of sumsets of finite sets in a group, form an essential part of the toolkit of additive combinatorics. In recent years, probabilistic or entropic analogs of many of these inequalities were introduced. We study analogues of these sumset estimates in the context of convex geometry and Lebesgue measure on ${\mathbb R}^n$. First, we observe tha… ▽ More Sumset estimates, which provide bounds on the cardinality of sumsets of finite sets in a group, form an essential part of the toolkit of additive combinatorics. In recent years, probabilistic or entropic analogs of many of these inequalities were introduced. We study analogues of these sumset estimates in the context of convex geometry and Lebesgue measure on ${\mathbb R}^n$. First, we observe that, with respect to Minkowski summation, volume is supermodular to arbitrary order on the space of convex bodies. Second, we explore sharp constants in the convex geometry analogues of variants of the Plünnecke-Ruzsa inequalities. In the last section of the paper, we provide connections of these inequalities to the classical Rogers-Shephard inequality. △ Less

Submitted 3 June, 2022; originally announced June 2022.

MSC Class: 52A40; 52A39; 52A20; 39B62

Volumes of subset Minkowski sums and the Lyusternik region

Authors: Franck Barthe, Mokshay Madiman

Abstract: We begin a systematic study of the region of possible values of the volumes of Minkowski subset sums of a collection of $M$ compact sets in $\mathbb{R}^d$, which we call the Lyusternik region, and make some first steps towards describing it. Our main result is that a fractional generalization of the Brunn-Minkowski-Lyusternik inequality conjectured by Bobkov et al. (2011) holds in dimension 1. Eve… ▽ More We begin a systematic study of the region of possible values of the volumes of Minkowski subset sums of a collection of $M$ compact sets in $\mathbb{R}^d$, which we call the Lyusternik region, and make some first steps towards describing it. Our main result is that a fractional generalization of the Brunn-Minkowski-Lyusternik inequality conjectured by Bobkov et al. (2011) holds in dimension 1. Even though Fradelizi et al. (2016) showed that it fails in general dimension, we show that a variant does hold in any dimension. △ Less

Submitted 11 March, 2024; v1 submitted 13 December, 2021; originally announced December 2021.

Journal ref: Discrete and Computational Geometry, Vol. 71, No. 3, pp. 823-848, April 2024

Bernoulli sums and Rényi entropy inequalities

Authors: Mokshay Madiman, James Melbourne, Cyril Roberto

Abstract: We investigate the Rényi entropy of independent sums of integer valued random variables through Fourier theoretic means, and give sharp comparisons between the variance and the Rényi entropy, for Poisson-Bernoulli variables. As applications we prove that a discrete ``min-entropy power'' is super additive on independent variables up to a universal constant, and give new bounds on an entropic genera… ▽ More We investigate the Rényi entropy of independent sums of integer valued random variables through Fourier theoretic means, and give sharp comparisons between the variance and the Rényi entropy, for Poisson-Bernoulli variables. As applications we prove that a discrete ``min-entropy power'' is super additive on independent variables up to a universal constant, and give new bounds on an entropic generalization of the Littlewood-Offord problem that are sharp in the ``Poisson regime''. △ Less

Submitted 15 March, 2024; v1 submitted 1 March, 2021; originally announced March 2021.

Journal ref: Bernoulli, Vol. 29, No. 2, pp. 1578--1599, May 2023

Log-Hessian and Deviation Bounds for Markov Semi-Groups, and Regularization Effect in $L^1$

Authors: Nathael Gozlan, Xue-Mei Li, Mokshay Madiman, Cyril Roberto, Paul-Marie Samson

Abstract: It is well known that some important Markov semi-groups have a "regularization effect" -- as for example the hypercontractivity property of the noise operator on the Boolean hypercube or the Ornstein-Uhlenbeck semi-group on the real line, which applies to functions in $L^p$ for $p>1$. Talagrand had conjectured in 1989 that the noise operator on the Boolean hypercube has a further subtle regulariza… ▽ More It is well known that some important Markov semi-groups have a "regularization effect" -- as for example the hypercontractivity property of the noise operator on the Boolean hypercube or the Ornstein-Uhlenbeck semi-group on the real line, which applies to functions in $L^p$ for $p>1$. Talagrand had conjectured in 1989 that the noise operator on the Boolean hypercube has a further subtle regularization property for functions that are just integrable, but this conjecture remains open. Nonetheless, the Gaussian analogue of this conjecture was proven in recent years by Eldan-Lee and Lehec, by combining an inequality for the log-Hessian of the Ornstein-Uhlenbeck semi-group with a new deviation inequality for log-semi-convex functions under Gaussian measure. In this work, we explore the question of how much more general this phenomenon is. Specifically, our first goal is to explore the validity of both these ingredients for some diffusion semi-groups in $\mathbb{R}^n$, as well as for the $M/M/\infty$ queue on the non-negative integers and the Laguerre semi-groups on the positive real line. Our second goal is to prove a one-dimensional regularization effect for these settings, even in those cases where these ingredients are not valid. △ Less

Submitted 1 December, 2022; v1 submitted 25 July, 2019; originally announced July 2019.

Comments: 33 pages

Two remarks on generalized entropy power inequalities

Authors: Mokshay Madiman, Piotr Nayar, Tomasz Tkocz

Abstract: This note contributes to the understanding of generalized entropy power inequalities. Our main goal is to construct a counter-example regarding monotonicity and entropy comparison of weighted sums of independent identically distributed log-concave random variables. We also present a complex analogue of a recent dependent entropy power inequality of Hao and Jog, and give a very simple proof. This note contributes to the understanding of generalized entropy power inequalities. Our main goal is to construct a counter-example regarding monotonicity and entropy comparison of weighted sums of independent identically distributed log-concave random variables. We also present a complex analogue of a recent dependent entropy power inequality of Hao and Jog, and give a very simple proof. △ Less

Submitted 3 April, 2019; originally announced April 2019.

Comments: arXiv:1811.00345 is split into 2 papers, with this being one

MSC Class: 94A17

Journal ref: Geometric Aspects of Functional Analysis: Israel Seminar (GAFA) 2017-2019 Volume II, B. Klartag and E. Milman (eds.), Lecture Notes in Mathematics, vol. 2266, Springer, 2020

Sharp moment-entropy inequalities and capacity bounds for log-concave distributions

Authors: Mokshay Madiman, Piotr Nayar, Tomasz Tkocz

Abstract: We show that the uniform distribution minimizes entropy among all one-dimensional symmetric log-concave distributions with fixed variance, as well as various generalizations of this fact to Rényi entropies of orders less than 1 and with moment constraints involving $p$-th absolute moments with $p\leq 2$. As consequences, we give new capacity bounds for additive noise channels with symmetric log-co… ▽ More We show that the uniform distribution minimizes entropy among all one-dimensional symmetric log-concave distributions with fixed variance, as well as various generalizations of this fact to Rényi entropies of orders less than 1 and with moment constraints involving $p$-th absolute moments with $p\leq 2$. As consequences, we give new capacity bounds for additive noise channels with symmetric log-concave noises, as well as for timing channels involving positive signal and noise where the noise has a decreasing log-concave density. In particular, we show that the capacity of an additive noise channel with symmetric, log-concave noise under an average power constraint is at most 0.254 bits per channel use greater than the capacity of an additive Gaussian noise channel with the same noise power. Consequences for reverse entropy power inequalities and connections to the slicing problem in convex geometry are also discussed. △ Less

Submitted 12 October, 2020; v1 submitted 1 November, 2018; originally announced November 2018.

Comments: the original paper split into two: this one and arXiv:1904.02314; additional results added to this part

MSC Class: 94A17 (Primary); 60E15 (Secondary)

Journal ref: IEEE Trans. Inform. Theory, vol. 67, no. 1, pp. 81-94, 2021

An Exact Upper Bound on the $L^p$ Lebesgue Constant and The $\infty$-Rényi Entropy Power Inequality for Integer Valued Random Variables

Authors: Peng Xu, Mokshay Madiman, James Melbourne

Abstract: In this paper, we proved an exact asymptotically sharp upper bound of the $L^p$ Lebesgue Constant (i.e. the $L^p$ norm of Dirichlet kernel) for $p\ge 2$. As an application, we also verified the implication of a new $\infty $-Rényi entropy power inequality for integer valued random variables. In this paper, we proved an exact asymptotically sharp upper bound of the $L^p$ Lebesgue Constant (i.e. the $L^p$ norm of Dirichlet kernel) for $p\ge 2$. As an application, we also verified the implication of a new $\infty $-Rényi entropy power inequality for integer valued random variables. △ Less

Submitted 23 August, 2018; originally announced August 2018.

Comments: 13 pages, 2 figures

MSC Class: 26D15 (Primary) 42A16; 46B99; 60B99 (Secondary)

The Differential Entropy of Mixtures: New Bounds and Applications

Authors: James Melbourne, Saurav Talukdar, Shreyas Bhaban, Mokshay Madiman, Murti V. Salapaka

Abstract: Mixture distributions are extensively used as a modeling tool in diverse areas from machine learning to communications engineering to physics, and obtaining bounds on the entropy of probability distributions is of fundamental importance in many of these applications. This article provides sharp bounds on the entropy concavity deficit, which is the difference between the entropy of the mixture and… ▽ More Mixture distributions are extensively used as a modeling tool in diverse areas from machine learning to communications engineering to physics, and obtaining bounds on the entropy of probability distributions is of fundamental importance in many of these applications. This article provides sharp bounds on the entropy concavity deficit, which is the difference between the entropy of the mixture and the weighted sum of entropies of constituent components. Toward establishing lower and upper bounds on the concavity deficit, results that are of importance in their own right are obtained. In order to obtain nontrivial upper bounds, properties of the skew-divergence are developed and notions of "skew" $f$-divergences are introduced; a reverse Pinsker inequality and a bound on Jensen-Shannon divergence are obtained along the way. Complementary lower bounds are derived with special attention paid to the case that corresponds to independent summation of a continuous and a discrete random variable. Several applications of the bounds are delineated, including to mutual information of additive noise channels, thermodynamics of computation, and functional inequalities. △ Less

Submitted 22 April, 2020; v1 submitted 29 May, 2018; originally announced May 2018.

Journal ref: IEEE Transactions on Information Theory, vol. 68, no. 4, pp. 2123-2146, April 2022

Majorization and Rényi Entropy Inequalities via Sperner Theory

Authors: Mokshay Madiman, Liyao Wang, Jae Oh Woo

Abstract: A natural link between the notions of majorization and strongly Sperner posets is elucidated. It is then used to obtain a variety of consequences, including new Rényi entropy inequalities for sums of independent, integer-valued random variables. A natural link between the notions of majorization and strongly Sperner posets is elucidated. It is then used to obtain a variety of consequences, including new Rényi entropy inequalities for sums of independent, integer-valued random variables. △ Less

Submitted 13 November, 2018; v1 submitted 4 December, 2017; originally announced December 2017.

Comments: Introduction was completely rewritten and there are numerous corrections. Expansion of background on Sperner theory, and several references are added

Journal ref: Discrete Mathematics (AEGT 2017 Special issue edited by S. Cioaba, R. Coulter, E. Fiorini, Q. Xiang, F. Pfender), vol. 342, no. 10, pp. 2911-2923, October 2019

Entropy Inequalities for Sums in Prime Cyclic Groups

Authors: Mokshay Madiman, Liyao Wang, Jae Oh Woo

Abstract: Lower bounds for the Rényi entropies of sums of independent random variables taking values in cyclic groups of prime order under permutations are established. The main ingredients of our approach are extended rearrangement inequalities in prime cyclic groups building on Lev (2001), and notions of stochastic ordering. Several applications are developed, including to discrete entropy power inequalit… ▽ More Lower bounds for the Rényi entropies of sums of independent random variables taking values in cyclic groups of prime order under permutations are established. The main ingredients of our approach are extended rearrangement inequalities in prime cyclic groups building on Lev (2001), and notions of stochastic ordering. Several applications are developed, including to discrete entropy power inequalities, the Littlewood-Offord problem, and counting solutions of certain linear systems. △ Less

Submitted 26 November, 2020; v1 submitted 2 October, 2017; originally announced October 2017.

Comments: 25 pages

Journal ref: SIAM J. Discrete Math., 35(3), pp. 1628-1649, 2021

Deviation inequalities for convex functions motivated by the Talagrand conjecture

Authors: Nathael Gozlan, Mokshay Madiman, Cyril Roberto, Paul-Marie Samson

Abstract: Motivated by Talagrand's conjecture on regularization properties of the natural semigroup on the Boolean hypercube, and in particular its continuous analogue involving regularization properties of the Ornstein-Uhlenbeck semigroup acting on in-tegrable functions, we explore deviation inequalities for log-semiconvex functions under Gaussian measure. Motivated by Talagrand's conjecture on regularization properties of the natural semigroup on the Boolean hypercube, and in particular its continuous analogue involving regularization properties of the Ornstein-Uhlenbeck semigroup acting on in-tegrable functions, we explore deviation inequalities for log-semiconvex functions under Gaussian measure. △ Less

Submitted 27 June, 2017; originally announced June 2017.

Journal ref: Zapiski Nauchnykh Seminarov POMI, vol. 457, Special issue in memory of V. N. Sudakov, pp. 168-182, 2017. Translation reprinted in the Journal of Mathematical Sciences, vol. 238, pp. 453-462, Springer, 2019

Rogozin's convolution inequality for locally compact groups

Authors: Mokshay Madiman, James Melbourne, Peng Xu

Abstract: General extensions of an inequality due to Rogozin, concerning the essential supremum of a convolution of probability density functions on the real line, are obtained. While a weak version of the inequality is proved in the very general context of Polish $σ$-compact groups, particular attention is paid to the group \(\mathbb{R}^d\), where the result can combined with rearrangement inequalities for… ▽ More General extensions of an inequality due to Rogozin, concerning the essential supremum of a convolution of probability density functions on the real line, are obtained. While a weak version of the inequality is proved in the very general context of Polish $σ$-compact groups, particular attention is paid to the group \(\mathbb{R}^d\), where the result can combined with rearrangement inequalities for certain linear images for a strong generalization. As a consequence, we obtain a unification and sharpening of both the \(\infty\)-Renyi entropy power inequality for sums of independent random vectors, due to Bobkov and Chistyakov, and the bounds on marginals of projections of product measures due to Rudelson and Vershynin (matching and extending the sharp improvement of Livshyts, Paouris and Pivovarov). The proof is elementary and relies on a characterization of extreme points of a class of probability measures in the general setting of Polish measure spaces, as well as the development of a generalization of Ball's cube slicing bounds for products of \(d\)-dimensional Euclidean balls (where the "co-dimension 1" case had been recently settled by Brzezinski). △ Less

Submitted 1 May, 2017; originally announced May 2017.

The convexification effect of Minkowski summation

Authors: Matthieu Fradelizi, Mokshay Madiman, Arnaud Marsiglietti, Artem Zvavitch

Abstract: Let us define for a compact set $A \subset \mathbb{R}^n$ the sequence $$ A(k) = \left\{\frac{a_1+\cdots +a_k}{k}: a_1, \ldots, a_k\in A\right\}=\frac{1}{k}\Big(\underset{k\ {\rm times}}{\underbrace{A + \cdots + A}}\Big). $$ It was independently proved by Shapley, Folkman and Starr (1969) and by Emerson and Greenleaf (1969) that $A(k)$ approaches the convex hull of $A$ in the Hausdorff distance ind… ▽ More Let us define for a compact set $A \subset \mathbb{R}^n$ the sequence $$ A(k) = \left\{\frac{a_1+\cdots +a_k}{k}: a_1, \ldots, a_k\in A\right\}=\frac{1}{k}\Big(\underset{k\ {\rm times}}{\underbrace{A + \cdots + A}}\Big). $$ It was independently proved by Shapley, Folkman and Starr (1969) and by Emerson and Greenleaf (1969) that $A(k)$ approaches the convex hull of $A$ in the Hausdorff distance induced by the Euclidean norm as $k$ goes to $\infty$. We explore in this survey how exactly $A(k)$ approaches the convex hull of $A$, and more generally, how a Minkowski sum of possibly different compact sets approaches convexity, as measured by various indices of non-convexity. The non-convexity indices considered include the Hausdorff distance induced by any norm on $\mathbb{R}^n$, the volume deficit (the difference of volumes), a non-convexity index introduced by Schneider (1975), and the effective standard deviation or inner radius. After first clarifying the interrelationships between these various indices of non-convexity, which were previously either unknown or scattered in the literature, we show that the volume deficit of $A(k)$ does not monotonically decrease to 0 in dimension 12 or above, thus falsifying a conjecture of Bobkov et al. (2011), even though their conjecture is proved to be true in dimension 1 and for certain sets $A$ with special structure. On the other hand, Schneider's index possesses a strong monotonicity property along the sequence $A(k)$, and both the Hausdorff distance and effective standard deviation are eventually monotone (once $k$ exceeds $n$). Along the way, we obtain new inequalities for the volume of the Minkowski sum of compact sets, falsify a conjecture of Dyn and Farkhi (2004), demonstrate applications of our results to combinatorial discrepancy theory, and suggest some questions worthy of further investigation. △ Less

Submitted 16 May, 2018; v1 submitted 18 April, 2017; originally announced April 2017.

Comments: 60 pages, 7 figures. v2: Title changed. v3: Added Section 7.2 resolving Dyn-Farkhi conjecture

MSC Class: 60E15 11B13 (Primary); 94A17 60F15 (Secondary)

Journal ref: EMS Surveys in Mathematical Sciences, 2018

Combinatorial Entropy Power Inequalities: A Preliminary Study of the Stam region

Authors: Mokshay Madiman, Farhad Ghassemi

Abstract: We initiate the study of the Stam region, defined as the subset of the positive orthant in $\mathbb{R}^{2^n-1}$ that arises from considering entropy powers of subset sums of $n$ independent random vectors in a Euclidean space of finite dimension. We show that the class of fractionally superadditive set functions provides an outer bound to the Stam region, resolving a conjecture of A. R. Barron and… ▽ More We initiate the study of the Stam region, defined as the subset of the positive orthant in $\mathbb{R}^{2^n-1}$ that arises from considering entropy powers of subset sums of $n$ independent random vectors in a Euclidean space of finite dimension. We show that the class of fractionally superadditive set functions provides an outer bound to the Stam region, resolving a conjecture of A. R. Barron and the first author. On the other hand, the entropy power of a sum of independent random vectors is not supermodular in any dimension. We also develop some qualitative properties of the Stam region, showing for instance that its closure is a logarithmically convex cone. △ Less

Submitted 29 June, 2018; v1 submitted 4 April, 2017; originally announced April 2017.

Comments: 16 pages

Journal ref: IEEE Transactions on Information Theory, vol. 65, no. 3, pp. 1375-1386, March 2019

The norm of the Fourier transform on compact or discrete abelian groups

Authors: Mokshay Madiman, Peng Xu

Abstract: We calculate the norm of the Fourier operator from $L^p(X)$ to $L^q(\hat{X})$ when $X$ is an infinite locally compact abelian group that is, furthermore, compact or discrete. This subsumes the sharp Hausdorff-Young inequality on such groups. In particular, we identify the region in $(p,q)$-space where the norm is infinite, generalizing a result of Fournier, and setting up a contrast with the case… ▽ More We calculate the norm of the Fourier operator from $L^p(X)$ to $L^q(\hat{X})$ when $X$ is an infinite locally compact abelian group that is, furthermore, compact or discrete. This subsumes the sharp Hausdorff-Young inequality on such groups. In particular, we identify the region in $(p,q)$-space where the norm is infinite, generalizing a result of Fournier, and setting up a contrast with the case of finite abelian groups, where the norm was determined by Gilbert and Rzeszotnik. As an application, uncertainty principles on such groups expressed in terms of Rényi entropies are discussed. △ Less

Submitted 14 November, 2016; originally announced November 2016.

Comments: 36 pages

Journal ref: Journal of Fourier Analysis and Applications, vol. 26, article no. 37, 2020

Forward and Reverse Entropy Power Inequalities in Convex Geometry

Authors: Mokshay Madiman, James Melbourne, Peng Xu

Abstract: The entropy power inequality, which plays a fundamental role in information theory and probability, may be seen as an analogue of the Brunn-Minkowski inequality. Motivated by this connection to Convex Geometry, we survey various recent developments on forward and reverse entropy power inequalities not just for the Shannon-Boltzmann entropy but also more generally for Rényi entropy. In the process,… ▽ More The entropy power inequality, which plays a fundamental role in information theory and probability, may be seen as an analogue of the Brunn-Minkowski inequality. Motivated by this connection to Convex Geometry, we survey various recent developments on forward and reverse entropy power inequalities not just for the Shannon-Boltzmann entropy but also more generally for Rényi entropy. In the process, we discuss connections between the so-called functional (or integral) and probabilistic (or entropic) analogues of some classical inequalities in geometric functional analysis △ Less

Submitted 24 December, 2016; v1 submitted 14 April, 2016; originally announced April 2016.

Comments: 54 pages. Changes in v2: improved organization, cleaned up exposition, and numerous references added

Journal ref: Book chapter in "Convexity and Concentration", E. Carlen, M. Madiman, E. Werner (eds.), The IMA Volumes in Mathematics and its Applications, Vol. 161, pp. 427-485, Springer, 2017

A combinatorial approach to small ball inequalities for sums and differences

Authors: Jiange Li, Mokshay Madiman

Abstract: Small ball inequalities have been extensively studied in the setting of Gaussian processes and associated Banach or Hilbert spaces. In this paper, we focus on studying small ball probabilities for sums or differences of independent, identically distributed random elements taking values in very general sets. Depending on the setting--abelian or nonabelian groups, or vector spaces, or Banach spaces-… ▽ More Small ball inequalities have been extensively studied in the setting of Gaussian processes and associated Banach or Hilbert spaces. In this paper, we focus on studying small ball probabilities for sums or differences of independent, identically distributed random elements taking values in very general sets. Depending on the setting--abelian or nonabelian groups, or vector spaces, or Banach spaces--we provide a collection of inequalities relating different small ball probabilities that are sharp in many cases of interest. We prove these distribution-free probabilistic inequalities by showing that underlying them are inequalities of extremal combinatorial nature, related among other things to classical packing problems such as the kissing number problem. Applications are given to moment inequalities. △ Less

Submitted 15 June, 2018; v1 submitted 15 January, 2016; originally announced January 2016.

Comments: Typos and grammatical errors are corrected, Section 7.2 (Holder-type inequalities) and Section 7.3 (reverse Holder-type inequalities) in the old draft are combined into one Section 7.2 (Moment inequalities), since a simple proof of the main result of Section 7.3 (Theorem 7.7, old draft, i.e., Corollary 7.8, new draft) is found, Combin. Probab. Comput. 2018

MSC Class: Primary 60C05; Secondary 05D05; 60B15

Journal ref: Combin. Probab. Comput., 28(1):100-129, 2019

Do Minkowski averages get progressively more convex?

Authors: Matthieu Fradelizi, Mokshay Madiman, Arnaud Marsiglietti, Artem Zvavitch

Abstract: Let us define, for a compact set $A \subset \mathbb{R}^n$, the Minkowski averages of $A$: $$ A(k) = \left\{\frac{a_1+\cdots +a_k}{k} : a_1, \ldots, a_k\in A\right\}=\frac{1}{k}\Big(\underset{k\ {\rm times}}{\underbrace{A + \cdots + A}}\Big). $$ We study the monotonicity of the convergence of $A(k)$ towards the convex hull of $A$, when considering the Hausdorff distance, the volume deficit and a no… ▽ More Let us define, for a compact set $A \subset \mathbb{R}^n$, the Minkowski averages of $A$: $$ A(k) = \left\{\frac{a_1+\cdots +a_k}{k} : a_1, \ldots, a_k\in A\right\}=\frac{1}{k}\Big(\underset{k\ {\rm times}}{\underbrace{A + \cdots + A}}\Big). $$ We study the monotonicity of the convergence of $A(k)$ towards the convex hull of $A$, when considering the Hausdorff distance, the volume deficit and a non-convexity index of Schneider as measures of convergence. For the volume deficit, we show that monotonicity fails in general, thus disproving a conjecture of Bobkov, Madiman and Wang. For Schneider's non-convexity index, we prove that a strong form of monotonicity holds, and for the Hausdorff distance, we establish that the sequence is eventually nonincreasing. △ Less

Submitted 6 February, 2016; v1 submitted 5 December, 2015; originally announced December 2015.

Comments: 6 pages, including figures. Contains announcement of results that will be part of a more comprehensive, forthcoming paper. Version 2 corrects a typo

Journal ref: Comptes Rendus de l'Academies des Sciences Paris, Serie I Mathematique, vol. 354, no. 2, pp. 185-189, February 2016

Concentration of information content for convex measures

Authors: Matthieu Fradelizi, Jiange Li, Mokshay Madiman

Abstract: We establish sharp exponential deviation estimates of the information content as well as a sharp bound on the varentropy for the class of convex measures on Euclidean spaces. This generalizes a similar development for log-concave measures in the recent work of Fradelizi, Madiman and Wang (2016). In particular, our results imply that convex measures in high dimensions are concentrated in an annulus… ▽ More We establish sharp exponential deviation estimates of the information content as well as a sharp bound on the varentropy for the class of convex measures on Euclidean spaces. This generalizes a similar development for log-concave measures in the recent work of Fradelizi, Madiman and Wang (2016). In particular, our results imply that convex measures in high dimensions are concentrated in an annulus between two convex sets (as in the log-concave case) despite their possibly having much heavier tails. Various tools and consequences are developed, including a sharp comparison result for Rényi entropies, inequalities of Kahane-Khinchine type for convex measures that extend those of Koldobsky, Pajor and Yaskin (2008) for log-concave measures, and an extension of Berwald's inequality (1947). △ Less

Submitted 23 February, 2020; v1 submitted 4 December, 2015; originally announced December 2015.

Comments: Added some references

Journal ref: Electron. J. Probab. 25(20):1-22, 2020

Optimal Concentration of Information Content For Log-Concave Densities

Authors: Matthieu Fradelizi, Mokshay Madiman, Liyao Wang

Abstract: An elementary proof is provided of sharp bounds for the varentropy of random vectors with log-concave densities, as well as for deviations of the information content from its mean. These bounds significantly improve on the bounds obtained by Bobkov and Madiman ({\it Ann. Probab.}, 39(4):1528--1543, 2011). An elementary proof is provided of sharp bounds for the varentropy of random vectors with log-concave densities, as well as for deviations of the information content from its mean. These bounds significantly improve on the bounds obtained by Bobkov and Madiman ({\it Ann. Probab.}, 39(4):1528--1543, 2011). △ Less

Submitted 26 October, 2015; v1 submitted 17 August, 2015; originally announced August 2015.

Comments: 15 pages. Changes in v2: Remark 2.5 (due to C. Saroglou) added with more general sufficient conditions for equality in Theorem 2.3. Also some minor corrections and added references

Journal ref: C. Houdré et al. (eds.), High Dimensional Probability VII, Progress in Probability 71, Springer, 2016

Entropy bounds on abelian groups and the Ruzsa divergence

Authors: Mokshay Madiman, Ioannis Kontoyiannis

Abstract: Over the past few years, a family of interesting new inequalities for the entropies of sums and differences of random variables has been developed by Ruzsa, Tao and others, motivated by analogous results in additive combinatorics. The present work extends these earlier results to the case of random variables taking values in $\mathbb{R}^n$ or, more generally, in arbitrary locally compact and Polis… ▽ More Over the past few years, a family of interesting new inequalities for the entropies of sums and differences of random variables has been developed by Ruzsa, Tao and others, motivated by analogous results in additive combinatorics. The present work extends these earlier results to the case of random variables taking values in $\mathbb{R}^n$ or, more generally, in arbitrary locally compact and Polish abelian groups. We isolate and study a key quantity, the Ruzsa divergence between two probability distributions, and we show that its properties can be used to extend the earlier inequalities to the present general setting. The new results established include several variations on the theme that the entropies of the sum and the difference of two independent random variables severely constrain each other. Although the setting is quite general, the result are already of interest (and new) for random vectors in $\mathbb{R}^n$. In that special case, quantitative bounds are provided for the stability of the equality conditions in the entropy power inequality; a reverse entropy power inequality for log-concave random vectors is proved; an information-theoretic analog of the Rogers-Shephard inequality for convex bodies is established; and it is observed that some of these results lead to new inequalities for the determinants of positive-definite matrices. Moreover, by considering the multiplicative subgroups of the complex plane, one obtains new inequalities for the differential entropies of products and ratios of nonzero, complex-valued random variables. △ Less

Submitted 26 October, 2015; v1 submitted 17 August, 2015; originally announced August 2015.

Comments: 26 pages. Changes in v2: Added Section V on entropies of products of random variables, corrected several typos, added some references

Journal ref: IEEE Transactions on Information Theory, vol. 64, no. 1, pp. 77-92, January 2018

Beyond the entropy power inequality, via rearrangements

Authors: Liyao Wang, Mokshay Madiman

Abstract: A lower bound on the Rényi differential entropy of a sum of independent random vectors is demonstrated in terms of rearrangements. For the special case of Boltzmann-Shannon entropy, this lower bound is better than that given by the entropy power inequality. Several applications are discussed, including a new proof of the classical entropy power inequality and an entropy inequality involving symmet… ▽ More A lower bound on the Rényi differential entropy of a sum of independent random vectors is demonstrated in terms of rearrangements. For the special case of Boltzmann-Shannon entropy, this lower bound is better than that given by the entropy power inequality. Several applications are discussed, including a new proof of the classical entropy power inequality and an entropy inequality involving symmetrization of Lévy processes. △ Less

Submitted 19 May, 2014; v1 submitted 23 July, 2013; originally announced July 2013.

Comments: 32 pages. v2: Several minor edits + Section X added

Journal ref: IEEE Transactions on Information Theory, vol. 60, no. 9, pp. 5116-5137, September 2014

Contribution to the theory of Pitman estimators

Authors: Abram M. Kagan, Tinghui Yu, Andrew Barron, Mokshay Madiman

Abstract: New inequalities are proved for the variance of the Pitman estimators (minimum variance equivariant estimators) of θconstructed from samples of fixed size from populations F(x-θ). The inequalities are closely related to the classical Stam inequality for the Fisher information, its analog in small samples, and a powerful variance drop inequality. The only condition required is finite variance of F;… ▽ More New inequalities are proved for the variance of the Pitman estimators (minimum variance equivariant estimators) of θconstructed from samples of fixed size from populations F(x-θ). The inequalities are closely related to the classical Stam inequality for the Fisher information, its analog in small samples, and a powerful variance drop inequality. The only condition required is finite variance of F; even the absolute continuity of F is not assumed. As corollaries of the main inequalities for small samples, one obtains alternate proofs of known properties of the Fisher information, as well as interesting new observations like the fact that the variance of the Pitman estimator based on a sample of size n scaled by n monotonically decreases in n. Extensions of the results to the polynomial versions of the Pitman estimators and a multivariate location parameter are given. Also, the search for characterization of equality conditions for one of the inequalities leads to a Cauchy-type functional equation for independent random variables, and an interesting new behavior of its solutions is described. △ Less

Submitted 13 February, 2013; originally announced February 2013.

Journal ref: Zapiski Nauchnykh Seminarov POMI, Special issue in honour of I. A. Ibragimov's 80th birthday, Vol. 408, pp. 245-267, 2012

Sumset and Inverse Sumset Inequalities for Differential Entropy and Mutual Information

Authors: Ioannis Kontoyiannis, Mokshay Madiman

Abstract: The sumset and inverse sumset theories of Freiman, Plünnecke and Ruzsa, give bounds connecting the cardinality of the sumset $A+B=\{a+b\;;\;a\in A,\,b\in B\}$ of two discrete sets $A,B$, to the cardinalities (or the finer structure) of the original sets $A,B$. For example, the sum-difference bound of Ruzsa states that, $|A+B|\,|A|\,|B|\leq|A-B|^3$, where the difference set… ▽ More The sumset and inverse sumset theories of Freiman, Plünnecke and Ruzsa, give bounds connecting the cardinality of the sumset $A+B=\{a+b\;;\;a\in A,\,b\in B\}$ of two discrete sets $A,B$, to the cardinalities (or the finer structure) of the original sets $A,B$. For example, the sum-difference bound of Ruzsa states that, $|A+B|\,|A|\,|B|\leq|A-B|^3$, where the difference set $A-B= \{a-b\;;\;a\in A,\,b\in B\}$. Interpreting the differential entropy $h(X)$ of a continuous random variable $X$ as (the logarithm of) the size of the effective support of $X$, the main contribution of this paper is a series of natural information-theoretic analogs for these results. For example, the Ruzsa sum-difference bound becomes the new inequality, $h(X+Y)+h(X)+h(Y)\leq 3h(X-Y)$, for any pair of independent continuous random variables $X$ and $Y$. Our results include differential-entropy versions of Ruzsa's triangle inequality, the Plünnecke-Ruzsa inequality, and the Balog-Szemerédi-Gowers lemma. Also we give a differential entropy version of the Freiman-Green-Ruzsa inverse-sumset theorem, which can be seen as a quantitative converse to the entropy power inequality. Versions of most of these results for the discrete entropy $H(X)$ were recently proved by Tao, relying heavily on a strong, functional form of the submodularity property of $H(X)$. Since differential entropy is {\em not} functionally submodular, in the continuous case many of the corresponding discrete proofs fail, in many cases requiring substantially new proof strategies. We find that the basic property that naturally replaces the discrete functional submodularity, is the data processing property of mutual information. △ Less

Submitted 3 June, 2012; originally announced June 2012.

Comments: 23 pages

Journal ref: IEEE Transactions on Information Theory, vol. 60, no. 8, pp. 4503-4514, August 2014

On the problem of reversibility of the entropy power inequality

Authors: Sergey G. Bobkov, Mokshay M. Madiman

Abstract: As was shown recently by the authors, the entropy power inequality can be reversed for independent summands with sufficiently concave densities, when the distributions of the summands are put in a special position. In this note it is proved that reversibility is impossible over the whole class of convex probability distributions. Related phenomena for identically distributed summands are also disc… ▽ More As was shown recently by the authors, the entropy power inequality can be reversed for independent summands with sufficiently concave densities, when the distributions of the summands are put in a special position. In this note it is proved that reversibility is impossible over the whole class of convex probability distributions. Related phenomena for identically distributed summands are also discussed. △ Less

Submitted 29 November, 2011; originally announced November 2011.

Comments: 13 pages

Journal ref: "Limit Theorems in Probability, Statistics and Number Theory (in honor of Friedrich Götze)", P. Eichelsbacher et al. (ed.), Springer Proceedings in Mathematics and Statistics 42, pp. 61-74, Springer-Verlag, 2013

Reverse Brunn-Minkowski and reverse entropy power inequalities for convex measures

Authors: Sergey Bobkov, Mokshay Madiman

Abstract: We develop a reverse entropy power inequality for convex measures, which may be seen as an affine-geometric inverse of the entropy power inequality of Shannon and Stam. The specialization of this inequality to log-concave measures may be seen as a version of Milman's reverse Brunn-Minkowski inequality. The proof relies on a demonstration of new relationships between the entropy of high dimensional… ▽ More We develop a reverse entropy power inequality for convex measures, which may be seen as an affine-geometric inverse of the entropy power inequality of Shannon and Stam. The specialization of this inequality to log-concave measures may be seen as a version of Milman's reverse Brunn-Minkowski inequality. The proof relies on a demonstration of new relationships between the entropy of high dimensional random vectors and the volume of convex bodies, and on a study of effective supports of convex measures, both of which are of independent interest, as well as on Milman's deep technology of $M$-ellipsoids and on certain information-theoretic inequalities. As a by-product, we also give a continuous analogue of some Plünnecke-Ruzsa inequalities from additive combinatorics. △ Less

Submitted 24 September, 2011; originally announced September 2011.

Comments: 28 pages, revised version of a document submitted in October 2010

Journal ref: Journal of Functional Analysis, Vol. 262, no. 7, pp. 3309-3339, April 2012

Dimensional behaviour of entropy and information

Authors: Sergey Bobkov, Mokshay Madiman

Abstract: We develop an information-theoretic perspective on some questions in convex geometry, providing for instance a new equipartition property for log-concave probability measures, some Gaussian comparison results for log-concave measures, an entropic formulation of the hyperplane conjecture, and a new reverse entropy power inequality for log-concave measures analogous to V. Milman's reverse Brunn-Mink… ▽ More We develop an information-theoretic perspective on some questions in convex geometry, providing for instance a new equipartition property for log-concave probability measures, some Gaussian comparison results for log-concave measures, an entropic formulation of the hyperplane conjecture, and a new reverse entropy power inequality for log-concave measures analogous to V. Milman's reverse Brunn-Minkowski inequality. △ Less

Submitted 17 January, 2011; originally announced January 2011.

Comments: 6 pages

Journal ref: C. R. Acad. Sci. Paris, Ser. I 349 (2011), pp. 201--204

Concentration of the information in data with log-concave distributions

Authors: Sergey Bobkov, Mokshay Madiman

Abstract: A concentration property of the functional ${-}\log f(X)$ is demonstrated, when a random vector X has a log-concave density f on $\mathbb{R}^n$. This concentration property implies in particular an extension of the Shannon-McMillan-Breiman strong ergodic theorem to the class of discrete-time stochastic processes with log-concave marginals. A concentration property of the functional ${-}\log f(X)$ is demonstrated, when a random vector X has a log-concave density f on $\mathbb{R}^n$. This concentration property implies in particular an extension of the Shannon-McMillan-Breiman strong ergodic theorem to the class of discrete-time stochastic processes with log-concave marginals. △ Less

Submitted 19 November, 2012; v1 submitted 25 December, 2010; originally announced December 2010.

Comments: Published in at http://dx.doi.org/10.1214/10-AOP592 the Annals of Probability (http://www.imstat.org/aop/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AOP-AOP592

Journal ref: Annals of Probability 2011, Vol. 39, No. 4, 1528-1543

Fractional generalizations of Young and Brunn-Minkowski inequalities

Authors: Sergey Bobkov, Mokshay Madiman, Liyao Wang

Abstract: A generalization of Young's inequality for convolution with sharp constant is conjectured for scenarios where more than two functions are being convolved, and it is proven for certain parameter ranges. The conjecture would provide a unified proof of recent entropy power inequalities of Barron and Madiman, as well as of a (conjectured) generalization of the Brunn-Minkowski inequality. It is shown t… ▽ More A generalization of Young's inequality for convolution with sharp constant is conjectured for scenarios where more than two functions are being convolved, and it is proven for certain parameter ranges. The conjecture would provide a unified proof of recent entropy power inequalities of Barron and Madiman, as well as of a (conjectured) generalization of the Brunn-Minkowski inequality. It is shown that the generalized Brunn-Minkowski conjecture is true for convex sets; an application of this to the law of large numbers for random sets is described. △ Less

Submitted 1 January, 2011; v1 submitted 15 June, 2010; originally announced June 2010.

Comments: 19 pages, numerous typos corrected, exposition improved, and references added, but no other substantial changes

Journal ref: Contemporary Mathematics 545, American Mathematical Society, 2011, pp. 35-53

The entropy per coordinate of a random vector is highly constrained under convexity conditions

Authors: Sergey Bobkov, Mokshay Madiman

Abstract: The entropy per coordinate in a log-concave random vector of any dimension with given density at the mode is shown to have a range of just 1. Uniform distributions on convex bodies are at the lower end of this range, the distribution with i.i.d. exponentially distributed coordinates is at the upper end, and the normal is exactly in the middle. Thus in terms of the amount of randomness as measured… ▽ More The entropy per coordinate in a log-concave random vector of any dimension with given density at the mode is shown to have a range of just 1. Uniform distributions on convex bodies are at the lower end of this range, the distribution with i.i.d. exponentially distributed coordinates is at the upper end, and the normal is exactly in the middle. Thus in terms of the amount of randomness as measured by entropy per coordinate, any log-concave random vector of any dimension contains randomness that differs from that in the normal random variable with the same maximal density value by at most 1/2. As applications, we obtain an information-theoretic formulation of the famous hyperplane conjecture in convex geometry, entropy bounds for certain infinitely divisible distributions, and quantitative estimates for the behavior of the density at the mode on convolution. More generally, one may consider so-called convex or hyperbolic probability measures on Euclidean spaces; we give new constraints on entropy per coordinate for this class of measures, which generalize our results under the log-concavity assumption, expose the extremal role of multivariate Pareto-type distributions, and give some applications. △ Less

Submitted 24 December, 2010; v1 submitted 15 June, 2010; originally announced June 2010.

Comments: 15 pages, revised for IEEE Transactions on Information Theory

Journal ref: IEEE Transactions on Information Theory, Vol. 57, no. 8, pp. 4940-4954, August 2011

Compound Poisson Approximation via Information Functionals

Authors: A. D. Barbour, Oliver Johnson, Ioannis Kontoyiannis, Mokshay Madiman

Abstract: An information-theoretic development is given for the problem of compound Poisson approximation, which parallels earlier treatments for Gaussian and Poisson approximation. Let $P_{S_n}$ be the distribution of a sum $S_n=\Sumn Y_i$ of independent integer-valued random variables $Y_i$. Nonasymptotic bounds are derived for the distance between $P_{S_n}$ and an appropriately chosen compound Poisson la… ▽ More An information-theoretic development is given for the problem of compound Poisson approximation, which parallels earlier treatments for Gaussian and Poisson approximation. Let $P_{S_n}$ be the distribution of a sum $S_n=\Sumn Y_i$ of independent integer-valued random variables $Y_i$. Nonasymptotic bounds are derived for the distance between $P_{S_n}$ and an appropriately chosen compound Poisson law. In the case where all $Y_i$ have the same conditional distribution given $\{Y_i\neq 0\}$, a bound on the relative entropy distance between $P_{S_n}$ and the compound Poisson distribution is derived, based on the data-processing property of relative entropy and earlier Poisson approximation results. When the $Y_i$ have arbitrary distributions, corresponding bounds are derived in terms of the total variation distance. The main technical ingredient is the introduction of two "information functionals," and the analysis of their properties. These information functionals play a role analogous to that of the classical Fisher information in normal approximation. Detailed comparisons are made between the resulting inequalities and related bounds. △ Less

Submitted 21 April, 2010; originally announced April 2010.

Comments: 27 pages

Journal ref: Electronic Journal of Probability, Vol 15, Paper no. 42, pages 1344-1369, 2010

Log-concavity, ultra-log-concavity, and a maximum entropy property of discrete compound Poisson measures

Authors: Oliver Johnson, Ioannis Kontoyiannis, Mokshay Madiman

Abstract: Sufficient conditions are developed, under which the compound Poisson distribution has maximal entropy within a natural class of probability measures on the nonnegative integers. Recently, one of the authors [O. Johnson, {\em Stoch. Proc. Appl.}, 2007] used a semigroup approach to show that the Poisson has maximal entropy among all ultra-log-concave distributions with fixed mean. We show via a non… ▽ More Sufficient conditions are developed, under which the compound Poisson distribution has maximal entropy within a natural class of probability measures on the nonnegative integers. Recently, one of the authors [O. Johnson, {\em Stoch. Proc. Appl.}, 2007] used a semigroup approach to show that the Poisson has maximal entropy among all ultra-log-concave distributions with fixed mean. We show via a non-trivial extension of this semigroup approach that the natural analog of the Poisson maximum entropy property remains valid if the compound Poisson distributions under consideration are log-concave, but that it fails in general. A parallel maximum entropy result is established for the family of compound binomial measures. Sufficient conditions for compound distributions to be log-concave are discussed and applications to combinatorics are examined; new bounds are derived on the entropy of the cardinality of a random independent set in a claw-free graph, and a connection is drawn to Mason's conjecture for matroids. The present results are primarily motivated by the desire to provide an information-theoretic foundation for compound Poisson approximation and associated limit theorems, analogous to the corresponding developments for the central limit theorem and for Poisson approximation. Our results also demonstrate new links between some probabilistic methods and the combinatorial notions of log-concavity and ultra-log-concavity, and they add to the growing body of work exploring the applications of maximum entropy characterizations to problems in discrete mathematics. △ Less

Submitted 27 September, 2011; v1 submitted 3 December, 2009; originally announced December 2009.

Comments: 30 pages. This submission supersedes arXiv:0805.4112v1. Changes in v2: Updated references, typos corrected

MSC Class: 94A17; 60E07; 60E15

Journal ref: Discrete Applied Mathematics, vol 161/9, pages 1232-1250, 2013

Entropy and set cardinality inequalities for partition-determined functions

Authors: Mokshay Madiman, Adam Marcus, Prasad Tetali

Abstract: A new notion of partition-determined functions is introduced, and several basic inequalities are developed for the entropy of such functions of independent random variables, as well as for cardinalities of compound sets obtained using these functions. Here a compound set means a set obtained by varying each argument of a function of several variables over a set associated with that argument, where… ▽ More A new notion of partition-determined functions is introduced, and several basic inequalities are developed for the entropy of such functions of independent random variables, as well as for cardinalities of compound sets obtained using these functions. Here a compound set means a set obtained by varying each argument of a function of several variables over a set associated with that argument, where all the sets are subsets of an appropriate algebraic structure so that the function is well defined. On the one hand, the entropy inequalities developed for partition-determined functions imply entropic analogues of general inequalities of Plünnecke-Ruzsa type. On the other hand, the cardinality inequalities developed for compound sets imply several inequalities for sumsets, including for instance a generalization of inequalities proved by Gyarmati, Matolcsi and Ruzsa (2010). We also provide partial progress towards a conjecture of Ruzsa (2007) for sumsets in nonabelian groups. All proofs are elementary and rely on properly developing certain information-theoretic inequalities. △ Less

Submitted 9 August, 2011; v1 submitted 30 December, 2008; originally announced January 2009.

Comments: 26 pages. v2: Revised version incorporating referee feedback plus inclusion of some additional corollaries and discussion. v3: Final version with minor corrections. To appear in Random Structures and Algorithms

Journal ref: Random Structures and Algorithms, Vol. 40, pp. 399-424, 2012

Information Inequalities for Joint Distributions, with Interpretations and Applications

Authors: Mokshay Madiman, Prasad Tetali

Abstract: Upper and lower bounds are obtained for the joint entropy of a collection of random variables in terms of an arbitrary collection of subset joint entropies. These inequalities generalize Shannon's chain rule for entropy as well as inequalities of Han, Fujishige and Shearer. A duality between the upper and lower bounds for joint entropy is developed. All of these results are shown to be special c… ▽ More Upper and lower bounds are obtained for the joint entropy of a collection of random variables in terms of an arbitrary collection of subset joint entropies. These inequalities generalize Shannon's chain rule for entropy as well as inequalities of Han, Fujishige and Shearer. A duality between the upper and lower bounds for joint entropy is developed. All of these results are shown to be special cases of general, new results for submodular functions-- thus, the inequalities presented constitute a richly structured class of Shannon-type inequalities. The new inequalities are applied to obtain new results in combinatorics, such as bounds on the number of independent sets in an arbitrary graph and the number of zero-error source-channel codes, as well as new determinantal inequalities in matrix theory. A new inequality for relative entropies is also developed, along with interpretations in terms of hypothesis testing. Finally, revealing connections of the results to literature in economics, computer science, and physics are explored. △ Less

Submitted 3 August, 2009; v1 submitted 30 December, 2008; originally announced January 2009.

Comments: 15 pages, 1 figure. Originally submitted to the IEEE Transactions on Information Theory in May 2007, the current version incorporates reviewer comments including elimination of an error

Journal ref: IEEE Transactions on Information Theory, Vol. 56(6), pp. 2699-2713, June 2010

On the entropy and log-concavity of compound Poisson measures

Authors: Oliver Johnson, Ioannis Kontoyiannis, Mokshay Madiman

Abstract: Motivated, in part, by the desire to develop an information-theoretic foundation for compound Poisson approximation limit theorems (analogous to the corresponding developments for the central limit theorem and for simple Poisson approximation), this work examines sufficient conditions under which the compound Poisson distribution has maximal entropy within a natural class of probability measures… ▽ More Motivated, in part, by the desire to develop an information-theoretic foundation for compound Poisson approximation limit theorems (analogous to the corresponding developments for the central limit theorem and for simple Poisson approximation), this work examines sufficient conditions under which the compound Poisson distribution has maximal entropy within a natural class of probability measures on the nonnegative integers. We show that the natural analog of the Poisson maximum entropy property remains valid if the measures under consideration are log-concave, but that it fails in general. A parallel maximum entropy result is established for the family of compound binomial measures. The proofs are largely based on ideas related to the semigroup approach introduced in recent work by Johnson for the Poisson family. Sufficient conditions are given for compound distributions to be log-concave, and specific examples are presented illustrating all the above results. △ Less

Submitted 27 May, 2008; originally announced May 2008.

Report number: Superceded by arXiv:0912.0581 MSC Class: 62B10; 94A17

Generalized Entropy Power Inequalities and Monotonicity Properties of Information

Authors: Mokshay Madiman, Andrew Barron

Abstract: New families of Fisher information and entropy power inequalities for sums of independent random variables are presented. These inequalities relate the information in the sum of $n$ independent random variables to the information contained in sums over subsets of the random variables, for an arbitrary collection of subsets. As a consequence, a simple proof of the monotonicity of information in c… ▽ More New families of Fisher information and entropy power inequalities for sums of independent random variables are presented. These inequalities relate the information in the sum of $n$ independent random variables to the information contained in sums over subsets of the random variables, for an arbitrary collection of subsets. As a consequence, a simple proof of the monotonicity of information in central limit theorems is obtained, both in the setting of i.i.d. summands as well as in the more general setting of independent summands with variance-standardized sums. △ Less

Submitted 2 July, 2007; v1 submitted 10 May, 2006; originally announced May 2006.

Comments: 13 pages. Many minor modifications from first version, plus a section on refined results. This is almost but not exactly identical to the published version

ACM Class: E.4; G.3

Journal ref: IEEE Transactions on Information Theory, Vol. 53(7), pp. 2317-2329, July 2007

Measure Concentration for Compound Poisson Distributions

Authors: I. Kontoyiannis, M. Madiman

Abstract: We give a simple development of the concentration properties of compound Poisson measures on the nonnegative integers. A new modification of the Herbst argument is applied to an appropriate modified logarithmic-Sobolev inequality to derive new concentration bounds. When the measure of interest does not have finite exponential moments, these bounds exhibit optimal polynomial decay. Simple new pro… ▽ More We give a simple development of the concentration properties of compound Poisson measures on the nonnegative integers. A new modification of the Herbst argument is applied to an appropriate modified logarithmic-Sobolev inequality to derive new concentration bounds. When the measure of interest does not have finite exponential moments, these bounds exhibit optimal polynomial decay. Simple new proofs are also given for earlier results of Houdr{é} (2002) and Wu (2000). △ Less

Submitted 21 June, 2005; originally announced June 2005.

Comments: 12 pages

MSC Class: 60E07; 60E15; 46N30; 39B62

Journal ref: Electronic Communications in Probability, 11, pp. 45-57, 2006