Showing 1–31 of 31 results for author: Chi, Z

On a Variation of Gambler's Ruin Problem

Authors: Zhiyi Chi, Vladimir Pozdnyakov

Abstract: Assume that letters (from a finite alphabet) in a text form a Markov chain. We track two distinct words, $U$ and $D$. A gambler gains 1 point for each occurrence of $U$ (including overlapping occurrences) and loses 1 point for each occurrence of $D$ (also including overlapping occurrences). We determine the probability of gaining $A$ points before losing $B$ points, where $A$ and $B$ are integers.… ▽ More Assume that letters (from a finite alphabet) in a text form a Markov chain. We track two distinct words, $U$ and $D$. A gambler gains 1 point for each occurrence of $U$ (including overlapping occurrences) and loses 1 point for each occurrence of $D$ (also including overlapping occurrences). We determine the probability of gaining $A$ points before losing $B$ points, where $A$ and $B$ are integers. Additionally, we find the expected waiting time until one of the two events -- gaining $A$ points or losing $B$ points -- occurs. △ Less

Submitted 1 June, 2025; originally announced June 2025.

MSC Class: 60G42; 60J10

On a distinctive property of Fourier bases associated with $N$- Bernoulli Convolutions

Authors: Zi-Chao Chi, Xing-Gang He, Zhi-Yi Wu

Abstract: A distinctive problem of harmonic analysis on $\R$ with respect to a Borel probability measure $μ$ is identifying all $t\in\R$ such that both \[\left\{e^{-2πiλx}: λ\inΛ\right\}\quad\text{and}\quad \left\{e^{-2πiλx}: λ\in tΛ\right\}\] form orthonormal bases of the space $L^2(μ)$. Currently, this phenomenon has been observed only in certain singular measures. It is deeply connected to the convergenc… ▽ More A distinctive problem of harmonic analysis on $\R$ with respect to a Borel probability measure $μ$ is identifying all $t\in\R$ such that both \[\left\{e^{-2πiλx}: λ\inΛ\right\}\quad\text{and}\quad \left\{e^{-2πiλx}: λ\in tΛ\right\}\] form orthonormal bases of the space $L^2(μ)$. Currently, this phenomenon has been observed only in certain singular measures. It is deeply connected to the convergence of Mock Fourier series with respect to the aforementioned bases. In this paper, we apply classical number theory to solve the general conjecture and basic problems in this field within the setting of $N$-Bernoulli convolutions, which extend almost all known results and give some new ones. △ Less

Submitted 30 May, 2025; originally announced June 2025.

Multiple testing under negative dependence

Authors: Ziyu Chi, Aaditya Ramdas, Ruodu Wang

Abstract: The multiple testing literature has primarily dealt with three types of dependence assumptions between p-values: independence, positive regression dependence, and arbitrary dependence. In this paper, we provide what we believe are the first theoretical results under various notions of negative dependence (negative Gaussian dependence, negative regression dependence, negative association, negative… ▽ More The multiple testing literature has primarily dealt with three types of dependence assumptions between p-values: independence, positive regression dependence, and arbitrary dependence. In this paper, we provide what we believe are the first theoretical results under various notions of negative dependence (negative Gaussian dependence, negative regression dependence, negative association, negative orthant dependence and weak negative dependence). These include the Simes global null test and the Benjamini-Hochberg procedure, which are known experimentally to be anti-conservative under negative dependence. The anti-conservativeness of these procedures is bounded by factors smaller than that under arbitrary dependence (in particular, by factors independent of the number of hypotheses). We also provide new results about negatively dependent e-values, and provide several examples as to when negative dependence may arise. Our proofs are elementary and short, thus amenable to extensions. △ Less

Submitted 8 May, 2024; v1 submitted 19 December, 2022; originally announced December 2022.

Comments: 28 pages, 5 figures

Spherical harmonic analysis for multivariate stable distributions

Authors: Zhiyi Chi

Abstract: Series representations consisting of spherical harmonics are obtained for characteristic exponents and probability density functions of multivariate stable distributions under various conditions. A esult potentially applicable in a practical setting is that for any distribution with stability index not equal to 1 and with a polynomial spectral spherical density, the series representation converges… ▽ More Series representations consisting of spherical harmonics are obtained for characteristic exponents and probability density functions of multivariate stable distributions under various conditions. A esult potentially applicable in a practical setting is that for any distribution with stability index not equal to 1 and with a polynomial spectral spherical density, the series representation converges absolutely with all terms being calculable in closed form. Asymptotic expansions consisting of spherical harmonics are also considered for probability density functions. △ Less

Submitted 15 October, 2021; v1 submitted 9 October, 2020; originally announced October 2020.

MSC Class: Primary 60G51; Secondary 60E07

On a lattice characterization of finite soluble $PST$-groups

Authors: Zhang Chi, Alexander N. Skiba

Abstract: Let $\mathfrak{F}$ be a class of finite groups and $G$ a finite group. Let ${\cal L}_{\mathfrak{F}}(G)$ be the set of all subgroups $A$ of $G$ with $A^{G}/A_{G}\in \mathfrak{F}$. A chief factor $H/K$ of $G$ is $\mathfrak{F}$-central in $G$ if $(H/K)\rtimes (G/C_{G}(H/K)) \in\mathfrak{F}$. We study the structure of $G$ under the hypothesis that every chief factor of $G$ between $A_{G}$ and… ▽ More Let $\mathfrak{F}$ be a class of finite groups and $G$ a finite group. Let ${\cal L}_{\mathfrak{F}}(G)$ be the set of all subgroups $A$ of $G$ with $A^{G}/A_{G}\in \mathfrak{F}$. A chief factor $H/K$ of $G$ is $\mathfrak{F}$-central in $G$ if $(H/K)\rtimes (G/C_{G}(H/K)) \in\mathfrak{F}$. We study the structure of $G$ under the hypothesis that every chief factor of $G$ between $A_{G}$ and $A^{G}$ is $\mathfrak{F}$-central in $G$ for every subgroup $A\in {\cal L}_{\mathfrak{F}}(G)$. As an application, we prove that a finite soluble group $G$ is a $PST$-group if and only if $A^{G}/A_{G}\leq Z_{\infty}(G/A_{G})$ for every subgroup $A\in {\cal L}_{\mathfrak{N}}(G)$, where $\mathfrak{N}$ is the class of all nilpotent groups. △ Less

Submitted 14 April, 2019; originally announced April 2019.

MSC Class: (2010): 20D10; 20D15; 20D20; 20E15

Law of two-sided exit by a spectrally positive strictly stable process

Authors: Zhiyi Chi

Abstract: For a spectrally positive strictly stable process with index in (1,2), the paper obtains i) the density of the time when the process makes first exit from an interval by hitting the interval's lower end point before jumping over its upper end point, and ii) the joint distribution of the time, the undershoot, and the jump of the process when it makes first exit the other way around. For i), the den… ▽ More For a spectrally positive strictly stable process with index in (1,2), the paper obtains i) the density of the time when the process makes first exit from an interval by hitting the interval's lower end point before jumping over its upper end point, and ii) the joint distribution of the time, the undershoot, and the jump of the process when it makes first exit the other way around. For i), the density of the time of first exit is expressed as an infinite sum of functions, each the product of a polynomial and an exponential function, with all coefficients determined by the roots of a Mittag-Leffler function. For ii), conditional on the undershoot, the time and the jump of first exit are independent, and the marginal conditional densities of the time has similar features as i). △ Less

Submitted 19 June, 2018; originally announced June 2018.

MSC Class: 60G51 (Primary); 60E07 (Secondary)

Finite groups with Frobenius normalizer condition for non-normal primary subgroups

Authors: Zhang Chi, Wenbin Guo

Abstract: A finite group $P$ is said to be \emph{primary} if $|P|=p^{a}$ for some prime $p$. We say a primary subgroup $P$ of a finite group $G$ satisfies the \emph{Frobenius normalizer condition} in $G$ if $N_{G}(P)/C_{G}(P)$ is a $p$-group provided $P$ is $p$-group. In this paper, we determine the structure of a finite group $G$ in which every non-subnormal primary subgroup satisfies the Frobenius normali… ▽ More A finite group $P$ is said to be \emph{primary} if $|P|=p^{a}$ for some prime $p$. We say a primary subgroup $P$ of a finite group $G$ satisfies the \emph{Frobenius normalizer condition} in $G$ if $N_{G}(P)/C_{G}(P)$ is a $p$-group provided $P$ is $p$-group. In this paper, we determine the structure of a finite group $G$ in which every non-subnormal primary subgroup satisfies the Frobenius normalized condition. In particular, we prove that if every non-normal primary subgroup of $G$ satisfies the Frobenius condition, then $G/F(G)$ is cyclic and every maximal non-normal nilpotent subgroup $U$ of $G$ with $F(G)U=G$ is a Carter subgroup of $G$. △ Less

Submitted 10 June, 2018; originally announced June 2018.

Comments: arXiv admin note: text overlap with arXiv:1801.09235

One application of the $σ$-local formations of finite groups

Authors: Zhang Chi, Alexander N. Skiba

Abstract: Throughout this paper, all groups are finite. Let $σ=\{σ_{i} | i\in I \}$ be some partition of the set of all primes $\Bbb{P}$. If $n$ is an integer, the symbol $σ(n)$ denotes the set $\{σ_{i} |σ_{i}\cap π(n)\ne \emptyset \}$. The integers $n$ and $m$ are called $σ$-coprime if $σ(n)\cap σ(m)=\emptyset$. Let $t > 1$ be a natural number and let $\mathfrak{F}$ be a class of groups. Then we sa… ▽ More Throughout this paper, all groups are finite. Let $σ=\{σ_{i} | i\in I \}$ be some partition of the set of all primes $\Bbb{P}$. If $n$ is an integer, the symbol $σ(n)$ denotes the set $\{σ_{i} |σ_{i}\cap π(n)\ne \emptyset \}$. The integers $n$ and $m$ are called $σ$-coprime if $σ(n)\cap σ(m)=\emptyset$. Let $t > 1$ be a natural number and let $\mathfrak{F}$ be a class of groups. Then we say that $\mathfrak{F}$ is $Σ_{t}^σ$-closed provided $\mathfrak{F}$ contains each group $G$ with subgroups $A_{1}, \ldots , A_{t}\in \mathfrak{F}$ whose indices $|G:A_{1}|$, $\ldots$, $|G:A_{t}|$ are pairwise $σ$-coprime. In this paper, we study $Σ_{t}^σ$-closed classes of finite groups. △ Less

Submitted 12 April, 2018; originally announced April 2018.

Comments: 12 pages

MSC Class: (2010): 20D10; 20D15; 20D20

On one generalization of finite nilpotent groups

Authors: Zhang Chi, Alexander N. Skiba

Abstract: Let $σ=\{σ_{i} | i\in I\}$ be a partition of the set $\Bbb{P}$ of all primes and $G$ a finite group. A chief factor $H/K$ of $G$ is said to be $σ$-central if the semidirect product $(H/K)\rtimes (G/C_{G}(H/K))$ is a $σ_{i}$-group for some $i=i(H/K)$. $G$ is called $σ$-nilpotent if every chief factor of $G$ is $σ$-central. We say that $G$ is semi-$σ$-nilpotent (respectively weakly semi-$σ$-nilpoten… ▽ More Let $σ=\{σ_{i} | i\in I\}$ be a partition of the set $\Bbb{P}$ of all primes and $G$ a finite group. A chief factor $H/K$ of $G$ is said to be $σ$-central if the semidirect product $(H/K)\rtimes (G/C_{G}(H/K))$ is a $σ_{i}$-group for some $i=i(H/K)$. $G$ is called $σ$-nilpotent if every chief factor of $G$ is $σ$-central. We say that $G$ is semi-$σ$-nilpotent (respectively weakly semi-$σ$-nilpotent) if the normalizer $N_{G}(A)$ of every non-normal (respectively every non-subnormal) $σ$-nilpotent subgroup $A$ of $G$ is $σ$-nilpotent. In this paper we determine the structure of finite semi-$σ$-nilpotent and weakly semi-$σ$-nilpotent groups. △ Less

Submitted 28 January, 2018; originally announced January 2018.

Comments: 16 pages

MSC Class: (2010): 20D10; 20D15; 20D30

Law of the first passage triple of a spectrally positive strictly stable process

Authors: Zhiyi Chi

Abstract: For a spectrally positive and strictly stable process with index in (1,2), a series representation is obtained for the joint distribution of the "first passage triple" that consists of the time of first passage and the undershoot and the overshoot at first passage. The result leads to several corollaries, including 1) the joint law of the first passage triple and the pre-passage running supremum,… ▽ More For a spectrally positive and strictly stable process with index in (1,2), a series representation is obtained for the joint distribution of the "first passage triple" that consists of the time of first passage and the undershoot and the overshoot at first passage. The result leads to several corollaries, including 1) the joint law of the first passage triple and the pre-passage running supremum, and 2) at a fixed time point, the joint law of the process' value, running supremum, and the time of the running supremum. The representation can be decomposed as a sum of strictly positive functions that allows exact sampling of the first passage triple. △ Less

Submitted 2 April, 2019; v1 submitted 21 January, 2018; originally announced January 2018.

MSC Class: 60G51; 60E07

Exact sampling of first passage event of certain symmetric Levy processes with unbounded variation

Authors: Zhiyi Chi

Abstract: We show that exact sampling of the first passage event can be done for a Levy process with unbounded variation, if the process can be embedded in a subordinated standard Brownian motion. By sampling a series of first exit events of the Brownian motion and first passage events of the subordinator, the first passage event of interest can be obtained. The sampling of the first exit time and pre-exit… ▽ More We show that exact sampling of the first passage event can be done for a Levy process with unbounded variation, if the process can be embedded in a subordinated standard Brownian motion. By sampling a series of first exit events of the Brownian motion and first passage events of the subordinator, the first passage event of interest can be obtained. The sampling of the first exit time and pre-exit location of the Brownian motion may be of independent interest. △ Less

Submitted 20 June, 2016; originally announced June 2016.

MSC Class: Primary 60G51; Secondary 60E07

Strong renewal theorems with infinite mean beyond local large deviations

Authors: Zhiyi Chi

Abstract: Let $F$ be a distribution function on the line in the domain of attraction of a stable law with exponent $α\in(0,1/2]$. We establish the strong renewal theorem for a random walk $S_1,S_2,\ldots$ with step distribution $F$, by extending the large deviations approach in Doney [Probab. Theory Related Fileds 107 (1997) 451-465]. This is done by introducing conditions on $F$ that in general rule out lo… ▽ More Let $F$ be a distribution function on the line in the domain of attraction of a stable law with exponent $α\in(0,1/2]$. We establish the strong renewal theorem for a random walk $S_1,S_2,\ldots$ with step distribution $F$, by extending the large deviations approach in Doney [Probab. Theory Related Fileds 107 (1997) 451-465]. This is done by introducing conditions on $F$ that in general rule out local large deviations bounds of the type $\mathbb{P}\{S_n\in(x,x+h]\}=O(n)\overline{F}(x)/x$, hence are significantly weaker than the boundedness condition in Doney (1997). We also give applications of the results on ladder height processes and infinitely divisible distributions. △ Less

Submitted 28 May, 2015; originally announced May 2015.

Comments: Published at http://dx.doi.org/10.1214/14-AAP1029 in the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AAP-AAP1029

Journal ref: Annals of Applied Probability 2015, Vol. 25, No. 3, 1513-1539

Random reversible Markov matrices with tunable extremal eigenvalues

Authors: Zhiyi Chi

Abstract: Random sampling of large Markov matrices with a tunable spectral gap, a nonuniform stationary distribution, and a nondegenerate limiting empirical spectral distribution (ESD) is useful. Fix $c>0$ and $p>0$. Let $A_n$ be the adjacency matrix of a random graph following $\mathrm{G}(n, p/n)$, known as the Erdős-Rényi distribution. Add $c/n$ to each entry of $A_n$ and then normalize its rows. It is sh… ▽ More Random sampling of large Markov matrices with a tunable spectral gap, a nonuniform stationary distribution, and a nondegenerate limiting empirical spectral distribution (ESD) is useful. Fix $c>0$ and $p>0$. Let $A_n$ be the adjacency matrix of a random graph following $\mathrm{G}(n, p/n)$, known as the Erdős-Rényi distribution. Add $c/n$ to each entry of $A_n$ and then normalize its rows. It is shown that the resulting Markov matrix has the desired properties. Its ESD weakly converges in probability to a symmetric nondegenerate distribution, and its extremal eigenvalues, other than 1, fall in $[-1/\sqrt{1+c/k},-b]\cup [b,1/\sqrt{1+c/k}]$ for any $0< b < 1/\sqrt{1+c}$, where $k = \lfloor p \rfloor + 1$. Thus, for $p\in (0,1)$, the spectral gap tends to $1-1/\sqrt{1+c}$. △ Less

Submitted 8 September, 2015; v1 submitted 8 May, 2015; originally announced May 2015.

MSC Class: 60B20; 05C80

On Multivariate Strong Renewal Theorem

Authors: Zhiyi Chi

Abstract: This paper takes the so-called probabilistic approach to the Strong Renewal Theorem (SRT) for multivariate distributions in the domain of attraction of a stable law. A version of the SRT is obtained that allows any kind of lattice-nonlattice composition of a distribution. A general bound is derived to control the so-called "small-$n$ contribution", which arises from random walk paths that have a r… ▽ More This paper takes the so-called probabilistic approach to the Strong Renewal Theorem (SRT) for multivariate distributions in the domain of attraction of a stable law. A version of the SRT is obtained that allows any kind of lattice-nonlattice composition of a distribution. A general bound is derived to control the so-called "small-$n$ contribution", which arises from random walk paths that have a relatively small number of steps but make large cumulative moves. The asymptotic negligibility of the small-$n$ contribution is essential to the SRT. Applications of the SRT are given, including some that provide a unified treatment to known results but with substantially weaker assumptions. △ Less

Submitted 15 March, 2017; v1 submitted 5 December, 2014; originally announced December 2014.

Report number: TR 14-29 MSC Class: 60K05; 60F10

Integral criteria for Strong Renewal Theorems with infinite mean

Authors: Zhiyi Chi

Abstract: Let $F$ be a probability measure on $\mathbb{R}$ in the domain of attraction of a stable law with exponent $α\in (0, 1)$. We establish integral criteria on $F$ that significantly expand the probabilistic approach to Strong Renewal Theorems (SRTs). The criterion for $α\in (0,1/2]$ is much weaker than currently available ones and in some cases provides sufficient and necessary conditions for the SRT… ▽ More Let $F$ be a probability measure on $\mathbb{R}$ in the domain of attraction of a stable law with exponent $α\in (0, 1)$. We establish integral criteria on $F$ that significantly expand the probabilistic approach to Strong Renewal Theorems (SRTs). The criterion for $α\in (0,1/2]$ is much weaker than currently available ones and in some cases provides sufficient and necessary conditions for the SRT. The criterion for $α\in (1/2, 1)$ establishes the SRT in full generality and in a unified way, barring the Limit Local Theorems employed. As an application, for infinitely divisible $F$, an integral criterion on its Lévy measure is established for the SRT. As another application, for $F$ in the domain of attraction of a stable law without centering, an integral criterion on $F$ is established for the SRT for the ladder height process of a random walk with step distribution $F$. △ Less

Submitted 15 April, 2014; v1 submitted 20 December, 2013; originally announced December 2013.

MSC Class: 60K05; 60F10

Nonnormal small jump approximation of infinitely divisible distributions

Authors: Zhiyi Chi

Abstract: We consider a type of nonnormal approximation of infinitely divisible distributions that incorporates compound Poisson, Gamma, and normal distributions. The approximation relies on achieving higher orders of cumulant matching, to obtain higher rates of approximation error decay. The parameters of the approximation are easy to fix. The computational complexity of random sampling of the approximatin… ▽ More We consider a type of nonnormal approximation of infinitely divisible distributions that incorporates compound Poisson, Gamma, and normal distributions. The approximation relies on achieving higher orders of cumulant matching, to obtain higher rates of approximation error decay. The parameters of the approximation are easy to fix. The computational complexity of random sampling of the approximating distribution in many cases is of the same order as normal approximation. Error bounds in terms of total variance distance are derived. Both the univariate and the multivariate cases of the approximation are considered. △ Less

Submitted 23 April, 2013; originally announced April 2013.

Report number: TR 2013-13, Department of Statistics, University of Connecticut MSC Class: Primary 60E07; Secondary 60G51

On exact sampling of the first passage event of Levy process with infinite Levy measure and bounded variation

Authors: Zhiyi Chi

Abstract: We present an exact sampling method for the first passage event of a Levy process. The idea is to embed the process into another one whose first passage event can be sampled exactly, and then recover the part belonging to the former from the latter. The method is based on several distributional properties that appear to be new. We obtain general procedures to sample the first passage event of a su… ▽ More We present an exact sampling method for the first passage event of a Levy process. The idea is to embed the process into another one whose first passage event can be sampled exactly, and then recover the part belonging to the former from the latter. The method is based on several distributional properties that appear to be new. We obtain general procedures to sample the first passage event of a subordinator across a regular non-increasing boundary, and that of a process with infinite Levy measure, bounded variation, and suitable drift across a constant level or interval. We give examples of application to a rather wide variety of Levy measures. △ Less

Submitted 10 July, 2012; originally announced July 2012.

MSC Class: Primary 60G51; Secondary 60E07

Stochastic Lipschitz continuity for high dimensional Lasso with multiple linear covariate structures or hidden linear covariates

Authors: Zhiyi Chi

Abstract: Two extensions of generalized linear models are considered. In the first one, response variables depend on multiple linear combinations of covariates. In the second one, only response variables are observed while the linear covariates are missing. We derive stochastic Lipschitz continuity results for the loss functions involved in the regression problems and apply them to get bounds on estimation… ▽ More Two extensions of generalized linear models are considered. In the first one, response variables depend on multiple linear combinations of covariates. In the second one, only response variables are observed while the linear covariates are missing. We derive stochastic Lipschitz continuity results for the loss functions involved in the regression problems and apply them to get bounds on estimation error for Lasso. Multivariate comparison results on Rademacher complexity are obtained as tools to establish the stochastic Lipschitz continuity results. △ Less

Submitted 5 November, 2010; originally announced November 2010.

MSC Class: 62G08; 60E15

A local stochastic Lipschitz condition with application to Lasso for high dimensional generalized linear models

Authors: Zhiyi Chi

Abstract: For regularized estimation, the upper tail behavior of the random Lipschitz coefficient associated with empirical loss functions is known to play an important role in the error bound of Lasso for high dimensional generalized linear models. The upper tail behavior is known for linear models but much less so for nonlinear models. We establish exponential type inequalities for the upper tail of the c… ▽ More For regularized estimation, the upper tail behavior of the random Lipschitz coefficient associated with empirical loss functions is known to play an important role in the error bound of Lasso for high dimensional generalized linear models. The upper tail behavior is known for linear models but much less so for nonlinear models. We establish exponential type inequalities for the upper tail of the coefficient and illustrate an application of the results to Lasso likelihood estimation for high dimensional generalized linear models. △ Less

Submitted 6 September, 2010; originally announced September 2010.

MSC Class: 62G08; 60E15

On $\ell_1$-regularized estimation for nonlinear models that have sparse underlying linear structures

Authors: Zhiyi Chi

Abstract: In a recent work (arXiv:0910.2517), for nonlinear models with sparse underlying linear structures, we studied the error bounds of $\ell_0$-regularized estimation. In this note, we show that $\ell_1$-regularized estimation in some important cases can achieve the same order of error bounds as those in the aforementioned work. In a recent work (arXiv:0910.2517), for nonlinear models with sparse underlying linear structures, we studied the error bounds of $\ell_0$-regularized estimation. In this note, we show that $\ell_1$-regularized estimation in some important cases can achieve the same order of error bounds as those in the aforementioned work. △ Less

Submitted 25 November, 2009; originally announced November 2009.

MSC Class: 62G05 (Primary); 62J02 (Secondary)

$L_0$ regularized estimation for nonlinear models that have sparse underlying linear structures

Authors: Zhiyi Chi

Abstract: We study the estimation of $β$ for the nonlinear model $y = f(X\sp{\top}β) + ε$ when $f$ is a nonlinear transformation that is known, $β$ has sparse nonzero coordinates, and the number of observations can be much smaller than that of parameters ($n\ll p$). We show that in order to bound the $L_2$ error of the $L_0$ regularized estimator $\hatβ$, i.e., $\|\hatβ- β\|_2$, it is sufficient to establ… ▽ More We study the estimation of $β$ for the nonlinear model $y = f(X\sp{\top}β) + ε$ when $f$ is a nonlinear transformation that is known, $β$ has sparse nonzero coordinates, and the number of observations can be much smaller than that of parameters ($n\ll p$). We show that in order to bound the $L_2$ error of the $L_0$ regularized estimator $\hatβ$, i.e., $\|\hatβ- β\|_2$, it is sufficient to establish two conditions. Based on this, we obtain bounds of the $L_2$ error for (1) $L_0$ regularized maximum likelihood estimation (MLE) for exponential linear models and (2) $L_0$ regularized least square (LS) regression for the more general case where $f$ is analytic. For the analytic case, we rely on power series expansion of $f$, which requires taking into account the singularities of $f$. △ Less

Submitted 13 October, 2009; originally announced October 2009.

Effects of statistical dependence on multiple testing under a hidden Markov model

Authors: Zhiyi Chi

Abstract: The performance of multiple hypothesis testing is known to be affected by the statistical dependence among random variables involved. The mechanisms responsible for this, however, are not well understood. We study the effects of the dependence structure of a finite state hidden Markov model (HMM) on the likelihood ratios critical for optimal multiple testing on the hidden states. Various convergen… ▽ More The performance of multiple hypothesis testing is known to be affected by the statistical dependence among random variables involved. The mechanisms responsible for this, however, are not well understood. We study the effects of the dependence structure of a finite state hidden Markov model (HMM) on the likelihood ratios critical for optimal multiple testing on the hidden states. Various convergence results are obtained for the likelihood ratios as the observations of the HMM form an increasing long chain. Analytic expansions of the first and second order derivatives are obtained for the case of binary states, explicitly showing the effects of the parameters of the HMM on the likelihood ratios. △ Less

Submitted 9 March, 2011; v1 submitted 9 April, 2009; originally announced April 2009.

Comments: Published in at http://dx.doi.org/10.1214/10-AOS822 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AOS-AOS822

Journal ref: Annals of Statistics 2011, Vol. 39, No. 1, 439-473

Data volume and power of multiple tests with small sample size per null

Authors: Zhiyi Chi

Abstract: In multiple hypothesis testing, the volume of data, defined as the number of replications per null times the total number of nulls, usually defines the amount of resource required. On the other hand, power is an important measure of performance for multiple testing. Due to practical constraints, the number of replications per null may not be large enough in terms of the difference between false… ▽ More In multiple hypothesis testing, the volume of data, defined as the number of replications per null times the total number of nulls, usually defines the amount of resource required. On the other hand, power is an important measure of performance for multiple testing. Due to practical constraints, the number of replications per null may not be large enough in terms of the difference between false and true nulls. For the case where the population fraction of false nulls is constant, we show that, as the difference between false and true nulls becomes increasingly subtle while the number of replications per null cannot increase fast enough, (1) in order to have enough chance that the data to be collected will yield some trustworthy findings, as measured by a conditional version of the positive false discovery rate (pFDR), the volume of data has to grow at a rate much faster than in the case where the number of replications per null can be large enough, and (2) in order to control the pFDR asymptotically, power has to decay to 0 in a rate highly sensitive to rejection criterion and there is no asymptotically most powerful procedures among those that control the pFDR asymptotically at the same level. △ Less

Submitted 4 June, 2009; v1 submitted 22 October, 2008; originally announced October 2008.

Comments: Final version to be published in Communications in Statistics

MSC Class: 62G10; 62H15 (Primary); 62G20 (Secondary)

FDR control for multiple hypothesis testing on composite nulls

Authors: Zhiyi Chi

Abstract: Multiple hypothesis testing often involves composite nulls, i.e., nulls that are associated with two or more distributions. In many cases, it is reasonable to assume that there is a prior distribution on the distributions despite it is unknown. When the number of distributions under true nulls is finite, we show that under the above assumption, the false discover rate (FDR) can be controlled usi… ▽ More Multiple hypothesis testing often involves composite nulls, i.e., nulls that are associated with two or more distributions. In many cases, it is reasonable to assume that there is a prior distribution on the distributions despite it is unknown. When the number of distributions under true nulls is finite, we show that under the above assumption, the false discover rate (FDR) can be controlled using $p$-values computed under constraints imposed by the empirical distribution of the observations. Comparing to FDR control using $p$-values defined as maximum significance level over all null distributions, the proposed FDR control can have substantially more power. △ Less

Submitted 30 July, 2008; originally announced July 2008.

MSC Class: 62G10; 62H15; 62G20

On the performance of FDR control: Constraints and a partial solution

Authors: Zhiyi Chi

Abstract: The False Discovery Rate (FDR) paradigm aims to attain certain control on Type I errors with relatively high power for multiple hypothesis testing. The Benjamini--Hochberg (BH) procedure is a well-known FDR controlling procedure. Under a random effects model, we show that, in general, unlike the FDR, the positive FDR (pFDR) of the BH procedure cannot be controlled at an arbitrarily low level due… ▽ More The False Discovery Rate (FDR) paradigm aims to attain certain control on Type I errors with relatively high power for multiple hypothesis testing. The Benjamini--Hochberg (BH) procedure is a well-known FDR controlling procedure. Under a random effects model, we show that, in general, unlike the FDR, the positive FDR (pFDR) of the BH procedure cannot be controlled at an arbitrarily low level due to the limited evidence provided by the observations to separate false and true nulls. This results in a criticality phenomenon, which is characterized by a transition of the procedure's power from being positive to asymptotically 0 without any reduction in the pFDR, once the target FDR control level is below a positive critical value. To address the constraints on the power and pFDR control imposed by the criticality phenomenon, we propose a procedure which applies BH-type procedures at multiple locations in the domain of $p$-values. Both analysis and simulations show that the proposed procedure can attain substantially improved power and pFDR control. △ Less

Submitted 17 October, 2007; originally announced October 2007.

Comments: Published in at http://dx.doi.org/10.1214/009053607000000037 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AOS-AOS0232 MSC Class: 62G10; 62H15 (Primary) 60G35 (Secondary)

Journal ref: Annals of Statistics 2007, Vol. 35, No. 4, 1409-1431

On the asymptotic of likelihood ratios for self-normalized large deviations

Authors: Zhiyi Chi

Abstract: Motivated by multiple statistical hypothesis testing, we obtain the limit of likelihood ratio of large deviations for self-normalized random variables, specifically, the ratio of $P(\sqrt{n}(\bar X +d/n) \ge x_n V)$ to $P(\sqrt{n}\bar X \ge x_n V)$, as $n\toi$, where $\bar X$ and $V$ are the sample mean and standard deviation of iid $X_1, ..., X_n$, respectively, $d>0$ is a constant and… ▽ More Motivated by multiple statistical hypothesis testing, we obtain the limit of likelihood ratio of large deviations for self-normalized random variables, specifically, the ratio of $P(\sqrt{n}(\bar X +d/n) \ge x_n V)$ to $P(\sqrt{n}\bar X \ge x_n V)$, as $n\toi$, where $\bar X$ and $V$ are the sample mean and standard deviation of iid $X_1, ..., X_n$, respectively, $d>0$ is a constant and $x_n \toi$. We show that the limit can have a simple form $e^{d/z_0}$, where $z_0$ is the unique maximizer of $z f(x)$ with $f$ the density of $X_i$. The result is applied to derive the minimum sample size per test in order to control the error rate of multiple testing at a target level, when real signals are different from noise signals only by a small shift. △ Less

Submitted 30 January, 2008; v1 submitted 10 September, 2007; originally announced September 2007.

Comments: typos on pages 1, 3 and 8 of the same type: missing or extra \sqrt{n} in the expressions of probabilities of large deviations

MSC Class: 60F10 (Primary); 62H15 (Secondary)

Uniform convergence of exact large deviations for renewal reward processes

Authors: Zhiyi Chi

Abstract: Let (X_n,Y_n) be i.i.d. random vectors. Let W(x) be the partial sum of Y_n just before that of X_n exceeds x>0. Motivated by stochastic models for neural activity, uniform convergence of the form $\sup_{c\in I}|a(c,x)\operatorname {Pr}\{W(x)\gecx\}-1|=o(1)$, $x\to\infty$, is established for probabilities of large deviations, with a(c,x) a deterministic function and I an open interval. To obtain… ▽ More Let (X_n,Y_n) be i.i.d. random vectors. Let W(x) be the partial sum of Y_n just before that of X_n exceeds x>0. Motivated by stochastic models for neural activity, uniform convergence of the form $\sup_{c\in I}|a(c,x)\operatorname {Pr}\{W(x)\gecx\}-1|=o(1)$, $x\to\infty$, is established for probabilities of large deviations, with a(c,x) a deterministic function and I an open interval. To obtain this uniform exact large deviations principle (LDP), we first establish the exponentially fast uniform convergence of a family of renewal measures and then apply it to appropriately tilted distributions of X_n and the moment generating function of W(x). The uniform exact LDP is obtained for cases where X_n has a subcomponent with a smooth density and Y_n is not a linear transform of X_n. An extension is also made to the partial sum at the first exceedance time. △ Less

Submitted 31 July, 2007; originally announced July 2007.

Comments: Published at http://dx.doi.org/10.1214/105051607000000023 in the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AAP-AAP0224 MSC Class: 60F10 (Primary) 60G51 (Secondary)

Journal ref: Annals of Applied Probability 2007, Vol. 17, No. 3, 1019-1048

False discovery rate control with multivariate $p$-values

Authors: Zhiyi Chi

Abstract: Multivariate statistics are often available as well as necessary in hypothesis tests. We study how to use such statistics to control not only false discovery rate (FDR) but also positive FDR (pFDR) with good power. We show that FDR can be controlled through nested regions of multivariate $p$-values of test statistics. If the distributions of the test statistics are known, then the regions can be… ▽ More Multivariate statistics are often available as well as necessary in hypothesis tests. We study how to use such statistics to control not only false discovery rate (FDR) but also positive FDR (pFDR) with good power. We show that FDR can be controlled through nested regions of multivariate $p$-values of test statistics. If the distributions of the test statistics are known, then the regions can be constructed explicitly to achieve FDR control with maximum power among procedures satisfying certain conditions. On the other hand, our focus is where the distributions are only partially known. Under certain conditions, a type of nested regions are proposed and shown to attain (p)FDR control with asymptotically maximum power as the pFDR control level approaches its attainable limit. The procedure based on the nested regions is compared with those based on other nested regions that are easier to construct as well as those based on more straightforward combinations of the test statistics. △ Less

Submitted 21 May, 2008; v1 submitted 4 June, 2007; originally announced June 2007.

Comments: Published in at http://dx.doi.org/10.1214/07-EJS147 the Electronic Journal of Statistics (http://www.i-journals.org/ejs/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-EJS-EJS_2007_147 MSC Class: 62G10; 62H15 (Primary) 62G20 (Secondary)

Journal ref: Electronic Journal of Statistics 2008, Vol. 2, 368-411

Sample size and positive false discovery rate control for multiple testing

Authors: Zhiyi Chi

Abstract: The positive false discovery rate (pFDR) is a useful overall measure of errors for multiple hypothesis testing, especially when the underlying goal is to attain one or more discoveries. Control of pFDR critically depends on how much evidence is available from data to distinguish between false and true nulls. Oftentimes, as many aspects of the data distributions are unknown, one may not be able t… ▽ More The positive false discovery rate (pFDR) is a useful overall measure of errors for multiple hypothesis testing, especially when the underlying goal is to attain one or more discoveries. Control of pFDR critically depends on how much evidence is available from data to distinguish between false and true nulls. Oftentimes, as many aspects of the data distributions are unknown, one may not be able to obtain strong enough evidence from the data for pFDR control. This raises the question as to how much data are needed to attain a target pFDR level. We study the asymptotics of the minimum number of observations per null for the pFDR control associated with multiple Studentized tests and $F$ tests, especially when the differences between false nulls and true nulls are small. For Studentized tests, we consider tests on shifts or other parameters associated with normal and general distributions. For $F$ tests, we also take into account the effect of the number of covariates in linear regression. The results show that in determining the minimum sample size per null for pFDR control, higher order statistical properties of data are important, and the number of covariates is important in tests to detect regression effects. △ Less

Submitted 9 May, 2007; v1 submitted 8 March, 2007; originally announced March 2007.

Comments: Published at http://dx.doi.org/10.1214/07-EJS045 in the Electronic Journal of Statistics (http://www.i-journals.org/ejs/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-EJS-EJS_2007_45 MSC Class: 62G10; 62H15 (Primary) 60F10 (Secondary)

Journal ref: Electronic Journal of Statistics 2007, Vol. 1, 77-118

Limit laws of estimators for critical multi-type Galton-Watson processes

Authors: Zhiyi Chi

Abstract: We consider the asymptotics of various estimators based on a large sample of branching trees from a critical multi-type Galton-Watson process, as the sample size increases to infinity. The asymptotics of additive functions of trees, such as sizes of trees and frequencies of types within trees, a higher-order asymptotic of the ``relative frequency'' estimator of the left eigenvector of the mean m… ▽ More We consider the asymptotics of various estimators based on a large sample of branching trees from a critical multi-type Galton-Watson process, as the sample size increases to infinity. The asymptotics of additive functions of trees, such as sizes of trees and frequencies of types within trees, a higher-order asymptotic of the ``relative frequency'' estimator of the left eigenvector of the mean matrix, a higher-order joint asymptotic of the maximum likelihood estimators of the offspring probabilities and the consistency of an estimator of the right eigenvector of the mean matrix, are established. △ Less

Submitted 24 March, 2005; originally announced March 2005.

Comments: Published at http://dx.doi.org/10.1214/105051604000000521 in the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AAP-AAP032 MSC Class: 60J80 (Primary) 60F05 (Secondary)

Journal ref: Annals of Applied Probability 2004, Vol. 14, No. 4, 1992-2015

Large deviations for template matching between point processes

Authors: Zhiyi Chi

Abstract: We study the asymptotics related to the following matching criteria for two independent realizations of point processes X\sim X and Y\sim Y. Given l>0, X\cap [0,l) serves as a template. For each t>0, the matching score between the template and Y\cap [t,t+l) is a weighted sum of the Euclidean distances from y-t to the template over all y\in Y\cap [t,t+l). The template matching criteria are used i… ▽ More We study the asymptotics related to the following matching criteria for two independent realizations of point processes X\sim X and Y\sim Y. Given l>0, X\cap [0,l) serves as a template. For each t>0, the matching score between the template and Y\cap [t,t+l) is a weighted sum of the Euclidean distances from y-t to the template over all y\in Y\cap [t,t+l). The template matching criteria are used in neuroscience to detect neural activity with certain patterns. We first consider W_l(θ), the waiting time until the matching score is above a given threshold θ. We show that whether the score is scalar- or vector-valued, (1/l)\log W_l(θ) converges almost surely to a constant whose explicit form is available, when X is a stationary ergodic process and Y is a homogeneous Poisson point process. Second, as l\to\infty, a strong approximation for -\log [\Pr{W_l(θ)=0}] by its rate function is established, and in the case where X is sufficiently mixing, the rates, after being centered and normalized by \sqrtl, satisfy a central limit theorem and almost sure invariance principle. The explicit form of the variance of the normal distribution is given for the case where X is a homogeneous Poisson process as well. △ Less

Submitted 22 March, 2005; originally announced March 2005.

Comments: Published at http://dx.doi.org/10.1214/105051604000000576 in the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AAP-AAP044 MSC Class: 60F10 (Primary) 60G55 (Secondary)

Journal ref: Annals of Applied Probability 2005, Vol. 15, No. 1A, 153-174