Showing 1–16 of 16 results for author: Skorski, M

Modelling 1/f Noise in TRNGs via Fractional Brownian Motion

Authors: Maciej Skorski

Abstract: Security of oscillatory true random number generators remains not fully understood due to insufficient understanding of complex $1/f^α$ phase noise. To bridge this gap, we introduce fractional Brownian motion as a comprehensive theoretical framework, capturing power-law spectral densities from white to flicker frequency noise. Our key contributions provide closed-form tractable solutions: (1) a… ▽ More Security of oscillatory true random number generators remains not fully understood due to insufficient understanding of complex $1/f^α$ phase noise. To bridge this gap, we introduce fractional Brownian motion as a comprehensive theoretical framework, capturing power-law spectral densities from white to flicker frequency noise. Our key contributions provide closed-form tractable solutions: (1) a quasi-renewal property showing conditional variance grows with power-law time dependence, enabling tractable leakage analysis; (2) closed-form min-entropy expressions under Gaussian phase posteriors; and (3) asymptotically unbiased Allan variance parameter estimation. This framework bridges physical modelling with cryptographic requirements, providing both theoretical foundations and practical calibration for oscillator-based TRNGs. △ Less

Submitted 28 May, 2025; v1 submitted 18 October, 2024; originally announced October 2024.

Comments: formal proofs, including bounded leakage conjecture

Quantifying Jitter Transfer for Differential Measurement: Enhancing Security of Oscillator-Based TRNGs

Authors: David Lubicz, Maciej Skorski

Abstract: The aim of this paper is to describe a way to improve the reliability of the measurement of the statistical parameters of the phase noise in a multi-ring oscillator-based TRNG. This is necessary to guarantee that the entropy rate is within the bounds prescribed by standards or security specifications. According to the literature, to filter out global noises which may strongly affect the measuremen… ▽ More The aim of this paper is to describe a way to improve the reliability of the measurement of the statistical parameters of the phase noise in a multi-ring oscillator-based TRNG. This is necessary to guarantee that the entropy rate is within the bounds prescribed by standards or security specifications. According to the literature, to filter out global noises which may strongly affect the measurement of the phase noise parameters, it is necessary to perform a differential measure. But a differential measurement only returns the parameters of the phase noise resulting of the composition of the noises of two oscillators whereas jitters parameters of individual oscillators are required to compute the entropy rate of a multi-ring oscillator-based TRNG. In this paper, we revisit the "jitter transfer principle" in conjunction with a tweaked design of an oscillator based TRNG to enjoy the precision of differential measures and, at the same time, obtain jitter parameters of individual oscillators. We show the relevance of our method with simulations and experiments with hardware implementations. △ Less

Submitted 10 October, 2024; originally announced October 2024.

Accurate Analysis of Sparse Random Projections

Authors: Maciej Skórski

Abstract: There has been recently a lot of research on sparse variants of random projections, faster adaptations of the state-of-the-art dimensionality reduction technique originally due to Johsnon and Lindenstrauss. Although the construction is very simple, its analyses are notoriously complicated. Meeting the demand for both simplicity and accuracy, this work establishes sharp sub-poissonian tail bounds f… ▽ More There has been recently a lot of research on sparse variants of random projections, faster adaptations of the state-of-the-art dimensionality reduction technique originally due to Johsnon and Lindenstrauss. Although the construction is very simple, its analyses are notoriously complicated. Meeting the demand for both simplicity and accuracy, this work establishes sharp sub-poissonian tail bounds for the distribution of sparse random projections. Compared to other works, this analysis provide superior numerical guarantees (exactly matching impossibility results) while being arguably less complicated (the technique resembles Bennet's Inequality and is of independent interest). △ Less

Submitted 3 July, 2024; originally announced July 2024.

Tight Chernoff-Like Bounds Under Limited Independence

Authors: Maciej Skorski

Abstract: This paper develops sharp bounds on moments of sums of k-wise independent bounded random variables, under constrained average variance. The result closes the problem addressed in part in the previous works of Schmidt et al. and Bellare, Rompel. We The work also discuss discusses other applications of independent interests, such as asymptotically sharp bounds on binomial moments. This paper develops sharp bounds on moments of sums of k-wise independent bounded random variables, under constrained average variance. The result closes the problem addressed in part in the previous works of Schmidt et al. and Bellare, Rompel. We The work also discuss discusses other applications of independent interests, such as asymptotically sharp bounds on binomial moments. △ Less

Submitted 4 September, 2022; originally announced September 2022.

Comments: The full version of the RANDOM'22 conference paper

On Missing Mass Variance

Authors: Maciej Skorski

Abstract: The missing mass refers to the probability of elements not observed in a sample, and since the work of Good and Turing during WWII, has been studied extensively in many areas including ecology, linguistic, networks and information theory. This work determines what is the \emph{maximal variance of the missing mass}, for any sample and alphabet sizes. The result helps in understanding the missing… ▽ More The missing mass refers to the probability of elements not observed in a sample, and since the work of Good and Turing during WWII, has been studied extensively in many areas including ecology, linguistic, networks and information theory. This work determines what is the \emph{maximal variance of the missing mass}, for any sample and alphabet sizes. The result helps in understanding the missing mass concentration properties. △ Less

Submitted 14 April, 2021; originally announced April 2021.

Bernstein-Type Bounds for Beta Distribution

Authors: Maciej Skorski

Abstract: This work obtains sharp closed-form exponential concentration inequalities of Bernstein type for the ubiquitous beta distribution, improving upon sub-gaussian and sub-gamma bounds previously studied in this context. The proof leverages a novel handy recursion of order 2 for central moments of the beta distribution, obtained from the hypergeometric representations of moments; this recursion is us… ▽ More This work obtains sharp closed-form exponential concentration inequalities of Bernstein type for the ubiquitous beta distribution, improving upon sub-gaussian and sub-gamma bounds previously studied in this context. The proof leverages a novel handy recursion of order 2 for central moments of the beta distribution, obtained from the hypergeometric representations of moments; this recursion is useful for obtaining explicit expressions for central moments and various tail approximations. △ Less

Submitted 6 February, 2023; v1 submitted 6 January, 2021; originally announced January 2021.

Comments: major revision - fixed a mistake in the proof

MSC Class: 60E05 ACM Class: G.3

Random Embeddings with Optimal Accuracy

Authors: Maciej Skorski

Abstract: This work constructs Jonson-Lindenstrauss embeddings with best accuracy, as measured by variance, mean-squared error and exponential concentration of the length distortion. Lower bounds for any data and embedding dimensions are determined, and accompanied by matching and efficiently samplable constructions (built on orthogonal matrices). Novel techniques: a unit sphere parametrization, the use of… ▽ More This work constructs Jonson-Lindenstrauss embeddings with best accuracy, as measured by variance, mean-squared error and exponential concentration of the length distortion. Lower bounds for any data and embedding dimensions are determined, and accompanied by matching and efficiently samplable constructions (built on orthogonal matrices). Novel techniques: a unit sphere parametrization, the use of singular-value latent variables and Schur-convexity are of independent interest. △ Less

Submitted 31 December, 2020; originally announced January 2021.

A Modern Analysis of Hutchinson's Trace Estimator

Authors: Maciej Skorski

Abstract: The paper establishes the new state-of-art in the accuracy analysis of Hutchinson's trace estimator. Leveraging tools that have not been previously used in this context, particularly hypercontractive inequalities and concentration properties of sub-gamma distributions, we offer an elegant and modular analysis, as well as numerically superior bounds. Besides these improvements, this work aims to be… ▽ More The paper establishes the new state-of-art in the accuracy analysis of Hutchinson's trace estimator. Leveraging tools that have not been previously used in this context, particularly hypercontractive inequalities and concentration properties of sub-gamma distributions, we offer an elegant and modular analysis, as well as numerically superior bounds. Besides these improvements, this work aims to better popularize the aforementioned techniques within the CS community. △ Less

Submitted 23 December, 2020; originally announced December 2020.

Handy Formulas for Binomial Moments

Authors: Maciej Skorski

Abstract: Despite the relevance of the binomial distribution for probability theory and applied statistical inference, its higher-order moments are poorly understood. The existing formulas are either not general enough, or not structured and simplified enough for intended applications. This paper introduces novel formulas for binomial moments, in form of \emph{polynomials in the variance} rather than in t… ▽ More Despite the relevance of the binomial distribution for probability theory and applied statistical inference, its higher-order moments are poorly understood. The existing formulas are either not general enough, or not structured and simplified enough for intended applications. This paper introduces novel formulas for binomial moments, in form of \emph{polynomials in the variance} rather than in the success probability. The obtained formulas are arguably better structured, simpler and superior in their numerical properties compared to prior works. In addition, the paper presents algorithms to derive these formulas along with working implementation in the Python symbolic algebra package. The novel approach is a combinatorial argument coupled with clever algebraic simplifications which rely on symmetrization theory. As an interesting byproduct we establish \emph{asymptotically sharp estimates for central binomial moments}, improving upon partial results from prior works. △ Less

Submitted 6 June, 2022; v1 submitted 11 December, 2020; originally announced December 2020.

MSC Class: 60E05: Distributions: general theory

Simple Analysis of Johnson-Lindenstrauss Transform under Neuroscience Constraints

Authors: Maciej Skorski

Abstract: The paper re-analyzes a version of the celebrated Johnson-Lindenstrauss Lemma, in which matrices are subjected to constraints that naturally emerge from neuroscience applications: a) sparsity and b) sign-consistency. This particular variant was studied first by Allen-Zhu, Gelashvili, Micali, Shavit and more recently by Jagadeesan (RANDOM'19). The contribution of this work is a novel proof, which… ▽ More The paper re-analyzes a version of the celebrated Johnson-Lindenstrauss Lemma, in which matrices are subjected to constraints that naturally emerge from neuroscience applications: a) sparsity and b) sign-consistency. This particular variant was studied first by Allen-Zhu, Gelashvili, Micali, Shavit and more recently by Jagadeesan (RANDOM'19). The contribution of this work is a novel proof, which in contrast to previous works a) uses the modern probability toolkit, particularly basics of sub-gaussian and sub-gamma estimates b) is self-contained, with no dependencies on subtle third-party results c) offers explicit constants. At the heart of our proof is a novel variant of Hanson-Wright Lemma (on concentration of quadratic forms). Of independent interest are also auxiliary facts on sub-gaussian random variables. △ Less

Submitted 20 August, 2020; originally announced August 2020.

Concentration Bounds for the Collision Estimator

Authors: Maciej Skorski

Abstract: We prove a strong concentration result about the natural collision estimator, which counts the number of collisions that occur within an iid sample. This estimator is at the heart of algorithms used for uniformity testing and entropy assessment. While the prior works were limited to only variance, we use elegant techniques of independent interest to bounds higher moments and conclude concentrati… ▽ More We prove a strong concentration result about the natural collision estimator, which counts the number of collisions that occur within an iid sample. This estimator is at the heart of algorithms used for uniformity testing and entropy assessment. While the prior works were limited to only variance, we use elegant techniques of independent interest to bounds higher moments and conclude concentration properties. As an immediate corollary we show that the estimator achieves high-probability guarantee on its own and there is no need for boosting (aka median/majority trick). △ Less

Submitted 25 June, 2020; v1 submitted 12 June, 2020; originally announced June 2020.

Comments: Fixing minor gaps in proofs

Revisiting Concentration of Missing Mass

Authors: Maciej Skorski

Abstract: We revisit the problem of \emph{missing mass concentration}, developing a new method of estimating concentration of heterogenic sums, in spirit of celebrated Rosenthal's inequality. As a result we slightly improve the state-of-art bounds due to Ben-Hamou at al., and simplify the proofs. We revisit the problem of \emph{missing mass concentration}, developing a new method of estimating concentration of heterogenic sums, in spirit of celebrated Rosenthal's inequality. As a result we slightly improve the state-of-art bounds due to Ben-Hamou at al., and simplify the proofs. △ Less

Submitted 22 May, 2020; v1 submitted 19 May, 2020; originally announced May 2020.

Comments: Added suplementary materials

Revisiting Initialization of Neural Networks

Authors: Maciej Skorski, Alessandro Temperoni, Martin Theobald

Abstract: The proper initialization of weights is crucial for the effective training and fast convergence of deep neural networks (DNNs). Prior work in this area has mostly focused on balancing the variance among weights per layer to maintain stability of (i) the input data propagated forwards through the network and (ii) the loss gradients propagated backwards, respectively. This prevalent heuristic is how… ▽ More The proper initialization of weights is crucial for the effective training and fast convergence of deep neural networks (DNNs). Prior work in this area has mostly focused on balancing the variance among weights per layer to maintain stability of (i) the input data propagated forwards through the network and (ii) the loss gradients propagated backwards, respectively. This prevalent heuristic is however agnostic of dependencies among gradients across the various layers and captures only firstorder effects. In this paper, we propose and discuss an initialization principle that is based on a rigorous estimation of the global curvature of weights across layers by approximating and controlling the norm of their Hessian matrix. The proposed approach is more systematic and recovers previous results for DNN activations such as smooth functions, dropouts, and ReLU. Our experiments on Word2Vec and the MNIST/CIFAR image classification tasks confirm that tracking the Hessian norm is a useful diagnostic tool which helps to more rigorously initialize weights △ Less

Submitted 4 June, 2020; v1 submitted 20 April, 2020; originally announced April 2020.

Missing Mass Concentration for Markov Chains

Authors: Maciej Skorski

Abstract: The problem of missing mass in statistical inference (posed by McAllester and Ortiz, NIPS'02; most recently revisited by Changa and Thangaraj, ISIT'2019) seeks to estimate the weight of symbols that have not been sampled yet from a source. So far all the approaches have been focused on the IID model which, although overly simplistic, is already not straightforward to tackle. The non-trivial part… ▽ More The problem of missing mass in statistical inference (posed by McAllester and Ortiz, NIPS'02; most recently revisited by Changa and Thangaraj, ISIT'2019) seeks to estimate the weight of symbols that have not been sampled yet from a source. So far all the approaches have been focused on the IID model which, although overly simplistic, is already not straightforward to tackle. The non-trivial part is in handling correlated events and sums of variables with very different scales where classical concentration inequalities do not yield good bounds. In this paper we develop the research on missing mass further, solving the problem for Markov chains. We reduce the problem to studying the tails of hitting times and finding \emph{log-additive approximations} to them. More precisely, we combine the technique of majorization and certain estimates on set hitting times to show how the problem can be eventually reduced back to the IID case. Our contribution are a) new technique to obtain missing mass bounds - we replace traditionally used negative association by majorization which works for a wider class of processes b) first (exponential) concentration bounds for missing mass in Markov chain models c) simplifications of recent results on set hitting times and d) simplified derivation of missing mass estimates for memory-less sources. △ Less

Submitted 14 January, 2020; v1 submitted 10 January, 2020; originally announced January 2020.

Comments: this subsumes the earlier draft

Kernel Density Estimation Bias under Minimal Assumptions

Authors: Maciej Skorski

Abstract: Kernel Density Estimation is a very popular technique of approximating a density function from samples. The accuracy is generally well-understood and depends, roughly speaking, on the kernel decay and local smoothness of the true density. However concrete statements in the literature are often invoked in very specific settings (simplified or overly conservative assumptions) or miss important but s… ▽ More Kernel Density Estimation is a very popular technique of approximating a density function from samples. The accuracy is generally well-understood and depends, roughly speaking, on the kernel decay and local smoothness of the true density. However concrete statements in the literature are often invoked in very specific settings (simplified or overly conservative assumptions) or miss important but subtle points (e.g. it is common to heuristically apply Taylor's expansion globally without referring to compactness). The contribution of this paper is twofold (a) we demonstrate that, when the bandwidth is an arbitrary invertible matrix going to zero, it is necessary to keep a certain balance between the \emph{kernel decay} and \emph{magnitudes of bandwidth eigenvalues}; in fact, without the sufficient decay the estimates may not be even bounded (b) we give a rigorous derivation of bounds with explicit constants for the bias, under possibly minimal assumptions. This connects the kernel decay, bandwidth norm, bandwidth determinant and density smoothness. It has been folklore that the issue with Taylor's formula can be fixed with more complicated assumptions on the density (for example p. 95 of "Kernel Smoothing" by Wand and Jones); we show that this is actually not necessary and can be handled by the kernel decay alone. △ Less

Submitted 2 January, 2019; originally announced January 2019.

Lower bounds on $q$-wise independence tails and applications to min-entropy condensers

Authors: Maciej Skorski

Abstract: We present novel and sharp lower bounds for higher load moments in the classical problem of mapping $M$ balls into $N$ bins by $q$-universal hashing, specialized to the case when $M=N$. As a corollary we prove a tight counterpart for the result about min-entropy condensers due to Dodis, Pietrzak and Wichs (CRYPTO'14), which has found important applications in key derivation. It states that condens… ▽ More We present novel and sharp lower bounds for higher load moments in the classical problem of mapping $M$ balls into $N$ bins by $q$-universal hashing, specialized to the case when $M=N$. As a corollary we prove a tight counterpart for the result about min-entropy condensers due to Dodis, Pietrzak and Wichs (CRYPTO'14), which has found important applications in key derivation. It states that condensing $k$ bits of min-entropy into a $k$-bit string $ε$-close to almost full min-entropy (precisely $ k-\log\log(1/ε)$ bits of entropy) can be achieved by the use of $q$-independent hashing with $q= \log(1/ε)$. We prove that when given a source of min-entropy $k$ and aiming at entropy loss $\ell = \log\log (1/ε) - 3$, the independence level $q=(1-o(1))\log(1/ε)$ is necessary (for small values of $ε$), which almost matches the positive result. Besides these asymptotic bounds, we provide clear hard bounds in terms of Bell numbers and some numerical examples. Our technique is based on an explicit representation of the load moments in terms of Stirling numbers, some asymptotic estimates on Stirling numbers and a tricky application of the Paley-Zygmund inequality. \keywords{ min-entropy condensers, key derivation, balls and bins hashing, anti-concentration inequalities } △ Less

Submitted 9 April, 2015; originally announced April 2015.