-
A new robust class of skew elliptical distributions
Authors:
H. Kwong,
S. Nadarajah
Abstract:
A new robust class of multivariate skew distributions is introduced. Practical aspects such as parameter estimation method of the proposed class are discussed, we show that the proposed class can be fitted under a reasonable time frame. Our study shows that the class of distributions is capable to model multivariate skewness structure and does not suffer from the curse of dimensionality as heavily…
▽ More
A new robust class of multivariate skew distributions is introduced. Practical aspects such as parameter estimation method of the proposed class are discussed, we show that the proposed class can be fitted under a reasonable time frame. Our study shows that the class of distributions is capable to model multivariate skewness structure and does not suffer from the curse of dimensionality as heavily as other distributions of similar complexity do, such as the class of canonical skew distributions. We also derive a nested form of the proposed class which appears to be the most flexible class of multivariate skew distributions in literature that has a closed-form density function. Numerical examples on two data sets, i) a data set containing daily river flow data recorded in the UK; and ii) a data set containing biomedical variables of athletes collected by the Australian Institute of Sports (AIS), are demonstrated. These examples further support the practicality of the proposed class on moderate dimensional data sets.
△ Less
Submitted 16 November, 2020; v1 submitted 3 November, 2020;
originally announced November 2020.
-
Self-guided Approximate Linear Programs
Authors:
Parshan Pakiman,
Selvaprabu Nadarajah,
Negar Soheili,
Qihang Lin
Abstract:
Approximate linear programs (ALPs) are well-known models based on value function approximations (VFAs) to obtain policies and lower bounds on the optimal policy cost of discounted-cost Markov decision processes (MDPs). Formulating an ALP requires (i) basis functions, the linear combination of which defines the VFA, and (ii) a state-relevance distribution, which determines the relative importance o…
▽ More
Approximate linear programs (ALPs) are well-known models based on value function approximations (VFAs) to obtain policies and lower bounds on the optimal policy cost of discounted-cost Markov decision processes (MDPs). Formulating an ALP requires (i) basis functions, the linear combination of which defines the VFA, and (ii) a state-relevance distribution, which determines the relative importance of different states in the ALP objective for the purpose of minimizing VFA error. Both these choices are typically heuristic: basis function selection relies on domain knowledge while the state-relevance distribution is specified using the frequency of states visited by a heuristic policy. We propose a self-guided sequence of ALPs that embeds random basis functions obtained via inexpensive sampling and uses the known VFA from the previous iteration to guide VFA computation in the current iteration. Self-guided ALPs mitigate the need for domain knowledge during basis function selection as well as the impact of the initial choice of the state-relevance distribution, thus significantly reducing the ALP implementation burden. We establish high probability error bounds on the VFAs from this sequence and show that a worst-case measure of policy performance is improved. We find that these favorable implementation and theoretical properties translate to encouraging numerical results on perishable inventory control and options pricing applications, where self-guided ALP policies improve upon policies from problem-specific methods. More broadly, our research takes a meaningful step toward application-agnostic policies and bounds for MDPs.
△ Less
Submitted 12 October, 2021; v1 submitted 8 January, 2020;
originally announced January 2020.
-
A Data Efficient and Feasible Level Set Method for Stochastic Convex Optimization with Expectation Constraints
Authors:
Qihang Lin,
Selvaprabu Nadarajah,
Negar Soheili,
Tianbao Yang
Abstract:
Stochastic convex optimization problems with expectation constraints (SOECs) are encountered in statistics and machine learning, business, and engineering. In data-rich environments, the SOEC objective and constraints contain expectations defined with respect to large datasets. Therefore, efficient algorithms for solving such SOECs need to limit the fraction of data points that they use, which we…
▽ More
Stochastic convex optimization problems with expectation constraints (SOECs) are encountered in statistics and machine learning, business, and engineering. In data-rich environments, the SOEC objective and constraints contain expectations defined with respect to large datasets. Therefore, efficient algorithms for solving such SOECs need to limit the fraction of data points that they use, which we refer to as algorithmic data complexity. Recent stochastic first order methods exhibit low data complexity when handling SOECs but guarantee near-feasibility and near-optimality only at convergence. These methods may thus return highly infeasible solutions when heuristically terminated, as is often the case, due to theoretical convergence criteria being highly conservative. This issue limits the use of first order methods in several applications where the SOEC constraints encode implementation requirements. We design a stochastic feasible level set method (SFLS) for SOECs that has low data complexity and emphasizes feasibility before convergence. Specifically, our level-set method solves a root-finding problem by calling a novel first order oracle that computes a stochastic upper bound on the level-set function by extending mirror descent and online validation techniques. We establish that SFLS maintains a high-probability feasible solution at each root-finding iteration and exhibits favorable iteration complexity compared to state-of-the-art deterministic feasible level set and stochastic subgradient methods. Numerical experiments on three diverse applications validate the low data complexity of SFLS relative to the former approach and highlight how SFLS finds feasible solutions with small optimality gaps significantly faster than the latter method.
△ Less
Submitted 1 January, 2020; v1 submitted 7 August, 2019;
originally announced August 2019.
-
An n-dimensional Rosenbrock Distribution for MCMC Testing
Authors:
Filippo Pagani,
Martin Wiegand,
Saralees Nadarajah
Abstract:
The Rosenbrock function is an ubiquitous benchmark problem for numerical optimisation, and variants have been proposed to test the performance of Markov Chain Monte Carlo algorithms. In this work we discuss the two-dimensional Rosenbrock density, its current $n$-dimensional extensions, and their advantages and limitations. We then propose a new extension to arbitrary dimensions called the Hybrid R…
▽ More
The Rosenbrock function is an ubiquitous benchmark problem for numerical optimisation, and variants have been proposed to test the performance of Markov Chain Monte Carlo algorithms. In this work we discuss the two-dimensional Rosenbrock density, its current $n$-dimensional extensions, and their advantages and limitations. We then propose a new extension to arbitrary dimensions called the Hybrid Rosenbrock distribution, which is composed of conditional normal kernels arranged in such a way that preserves the key features of the original kernel. Moreover, due to its structure, the Hybrid Rosenbrock distribution is analytically tractable and possesses several desirable properties, which make it an excellent test model for computational algorithms.
△ Less
Submitted 7 May, 2020; v1 submitted 22 March, 2019;
originally announced March 2019.
-
alphastable: An R Package for Modelling Multivariate Stable and Mixture of Symmetric Stable Distributions
Authors:
Mahdi Teimouri,
Mahdi Torshizi,
Adel Mohammadpour,
Saralees Nadarajah
Abstract:
The family of stable distributions received extensive applications in many fields of studies since it incorporates both the skewness and heavy tails. In this paper, we introduce a package written in the R language called alphastable. The alphastable performs a variety of tasks including: 1- generating random numbers from univariate, truncated, and multivariate stable distributions. 2- computing th…
▽ More
The family of stable distributions received extensive applications in many fields of studies since it incorporates both the skewness and heavy tails. In this paper, we introduce a package written in the R language called alphastable. The alphastable performs a variety of tasks including: 1- generating random numbers from univariate, truncated, and multivariate stable distributions. 2- computing the probability density function of univariate and multivariate elliptically contoured stable distributions, 3- computing the distribution function of univariate stable distributions, 4- estimating the parameters of univariate symmetric stable, univariate Cauchy, mixture of Cauchy, mixture of univariate symmetric stable, multivariate elliptically contoured stable, and multivariate strictly stable distributions. This package, as it will be shown, is very useful for modelling data in univariate and multivariate cases that arise in the fields of finance and economics.
△ Less
Submitted 25 September, 2018;
originally announced September 2018.
-
Rates of convergence of extremes from skew normal samples
Authors:
Xin Liao,
Zuoxiang Peng,
Saralees Nadarajah,
Xiaoqian Wang
Abstract:
For a skew normal random sequence, convergence rates of the distribution of its partial maximum to the Gumbel extreme value distribution are derived. The asymptotic expansion of the distribution of the normalized maximum is given under an optimal choice of norming constants. We find that the optimal convergence rate of the normalized maximum to the Gumbel extreme value distribution is proportional…
▽ More
For a skew normal random sequence, convergence rates of the distribution of its partial maximum to the Gumbel extreme value distribution are derived. The asymptotic expansion of the distribution of the normalized maximum is given under an optimal choice of norming constants. We find that the optimal convergence rate of the normalized maximum to the Gumbel extreme value distribution is proportional to $1/\log n$.
△ Less
Submitted 5 December, 2012;
originally announced December 2012.
-
The chain rule for functionals with applications to functions of moments
Authors:
C. S. Withers,
S. Nadarajah
Abstract:
The chain rule for derivatives of a function of a function is extended to a function of a statistical functional, and applied to obtain approximations to the cumulants, distribution and quantiles of functions of sample moments, and so to obtain third order confidence intervals and estimates of reduced bias for functions of moments. As an example we give the distribution of the standardized skewnes…
▽ More
The chain rule for derivatives of a function of a function is extended to a function of a statistical functional, and applied to obtain approximations to the cumulants, distribution and quantiles of functions of sample moments, and so to obtain third order confidence intervals and estimates of reduced bias for functions of moments. As an example we give the distribution of the standardized skewness for a normal sample to magnitude $O(n^{-2})$, where $n$ is the sample size.
△ Less
Submitted 1 November, 2012;
originally announced November 2012.
-
Expansions about the gamma for the distribution and quantiles of a standard estimate
Authors:
C. S. Withers,
S. Nadarajah
Abstract:
We give expansions for the distribution, density, and quantiles of an estimate, building on results of Cornish, Fisher, Hill, Davis and the authors. The estimate is assumed to be non-lattice with the standard expansions for its cumulants. By expanding about a skew variable with matched skewness, one can drastically reduce the number of terms needed for a given level of accuracy. The building block…
▽ More
We give expansions for the distribution, density, and quantiles of an estimate, building on results of Cornish, Fisher, Hill, Davis and the authors. The estimate is assumed to be non-lattice with the standard expansions for its cumulants. By expanding about a skew variable with matched skewness, one can drastically reduce the number of terms needed for a given level of accuracy. The building blocks generalize the Hermite polynomials. We demonstrate with expansions about the gamma.
△ Less
Submitted 15 October, 2012;
originally announced October 2012.
-
Accurate inference for a one parameter distribution based on the mean of a transformed sample
Authors:
C. S. Withers,
S. Nadarajah
Abstract:
A great deal of inference in statistics is based on making the approximation that a statistic is normally distributed. The error in doing so is generally $O(n^{-1/2})$ and can be very considerable when the distribution is heavily biased or skew. This note shows how one may reduce this error to $O(n^{-(j+1)/2})$, where $j$ is a given integer. The case considered is when the statistic is the mean of…
▽ More
A great deal of inference in statistics is based on making the approximation that a statistic is normally distributed. The error in doing so is generally $O(n^{-1/2})$ and can be very considerable when the distribution is heavily biased or skew. This note shows how one may reduce this error to $O(n^{-(j+1)/2})$, where $j$ is a given integer. The case considered is when the statistic is the mean of the sample values from a continuous one-parameter distribution, after the sample has undergone an initial transformation.
△ Less
Submitted 11 September, 2010;
originally announced September 2010.
-
The distribution and quantiles of functionals of weighted empirical distributions when observations have different distributions
Authors:
C. S. Withers,
S. Nadarajah
Abstract:
This paper extends Edgeworth-Cornish-Fisher expansions for the distribution and quantiles of nonparametric estimates in two ways. Firstly it allows observations to have different distributions. Secondly it allows the observations to be weighted in a predetermined way. The use of weighted estimates has a long history including applications to regression, rank statistics and Bayes theory. However,…
▽ More
This paper extends Edgeworth-Cornish-Fisher expansions for the distribution and quantiles of nonparametric estimates in two ways. Firstly it allows observations to have different distributions. Secondly it allows the observations to be weighted in a predetermined way. The use of weighted estimates has a long history including applications to regression, rank statistics and Bayes theory. However, asymptotic results have generally been only first order (the CLT and weak convergence). We give third order asymptotics for the distribution and percentiles of any smooth functional of a weighted empirical distribution, thus allowing a considerable increase in accuracy over earlier CLT results.
Consider independent non-identically distributed ({\it non-iid}) observations $X_{1n}, ..., X_{nn}$ in $R^s$. Let $\hat{F}(x)$ be their {\it weighted empirical distribution} with weights $w_{1n}, ..., w_{nn}$. We obtain cumulant expansions and hence Edgeworth-Cornish-Fisher expansions for $T(\hat{F})$ for any smooth functional $T(\cdot)$ by extending the concepts of von Mises derivatives to signed measures of total measure 1. As an example we give the cumulant coefficients needed for Edgeworth-Cornish-Fisher expansions to $O(n^{-3/2})$ for the sample variance when observations are non-iid.
△ Less
Submitted 23 February, 2010;
originally announced February 2010.
-
Expansions for Quantiles and Multivariate Moments of Extremes for Distributions of Pareto Type
Authors:
Saralees Nadarajah,
Christopher S. Withers
Abstract:
Let $X_{nr}$ be the $r$th largest of a random sample of size $n$ from a distribution $F (x) = 1 - \sum_{i = 0}^\infty c_i x^{-α- i β}$ for $α> 0$ and $β> 0$. An inversion theorem is proved and used to derive an expansion for the quantile $F^{-1} (u)$ and powers of it. From this an expansion in powers of $(n^{-1}, n^{-β/α})$ is given for the multivariate moments of the extremes…
▽ More
Let $X_{nr}$ be the $r$th largest of a random sample of size $n$ from a distribution $F (x) = 1 - \sum_{i = 0}^\infty c_i x^{-α- i β}$ for $α> 0$ and $β> 0$. An inversion theorem is proved and used to derive an expansion for the quantile $F^{-1} (u)$ and powers of it. From this an expansion in powers of $(n^{-1}, n^{-β/α})$ is given for the multivariate moments of the extremes $\{X_{n, n - s_i}, 1 \leq i \leq k \}/n^{1/α}$ for fixed ${\bf s} = (s_1, ..., s_k)$, where $k \geq 1$. Examples include the Cauchy, Student $t$, $F$, second extreme distributions and stable laws of index $α< 1$.
△ Less
Submitted 25 March, 2009;
originally announced March 2009.
-
Analytic Bias Reduction for $k$-Sample Functionals
Authors:
Christopher S. Withers,
Saralees Nadarajah
Abstract:
We give analytic methods for nonparametric bias reduction that remove the need for computationally intensive methods like the bootstrap and the jackknife.
We call an estimate {\it $p$th order} if its bias has magnitude $n_0^{-p}$ as $n_0 \to \infty$, where $n_0$ is the sample size (or the minimum sample size if the estimate is a function of more than one sample). Most estimates are only first…
▽ More
We give analytic methods for nonparametric bias reduction that remove the need for computationally intensive methods like the bootstrap and the jackknife.
We call an estimate {\it $p$th order} if its bias has magnitude $n_0^{-p}$ as $n_0 \to \infty$, where $n_0$ is the sample size (or the minimum sample size if the estimate is a function of more than one sample). Most estimates are only first order and require O(N) calculations, where $N$ is the total sample size. The usual bootstrap and jackknife estimates are second order but they are computationally intensive, requiring $O(N^2)$ calculations for one sample. By contrast Jaeckel's infinitesimal jackknife is an analytic second order one sample estimate requiring only O(N) calculations. When $p$th order bootstrap and jackknife estimates are available, they require $O(N^p)$ calculations, and so become even more computationally intensive if one chooses $p>2$.
For general $p$ we provide analytic $p$th order nonparametric estimates that require only O(N) calculations. Our estimates are given in terms of the von Mises derivatives of the functional being estimated, evaluated at the empirical distribution.
For products of moments an unbiased estimate exists: our form for this "polykay" is much simpler than the usual form in terms of power sums.
△ Less
Submitted 16 March, 2009;
originally announced March 2009.
-
Asymptotic tail properties of the distributions in the class of dispersion models
Authors:
Alexandre B. Simas,
Gauss M. Cordeiro,
Saralees Nadarajah
Abstract:
The class of dispersion models introduced by Jørgensen (1997b) covers many known distributions such as the normal, Student t, gamma, inverse Gaussian, hyperbola, von-Mises, among others. We study the small dispersion asymptotic (Jørgensen, 1987b) behavior of the probability density functions of dispersion models which satisfy the uniformly convergent saddlepoint approximation. Our results extend…
▽ More
The class of dispersion models introduced by Jørgensen (1997b) covers many known distributions such as the normal, Student t, gamma, inverse Gaussian, hyperbola, von-Mises, among others. We study the small dispersion asymptotic (Jørgensen, 1987b) behavior of the probability density functions of dispersion models which satisfy the uniformly convergent saddlepoint approximation. Our results extend those obtained by Finner et al. (2008).
△ Less
Submitted 10 September, 2008;
originally announced September 2008.
-
The distribution of the maximum of a first order moving average: the discrete case
Authors:
Christopher S. Withers,
Saralees Nadarajah
Abstract:
We give the distribution of $M_n$, the maximum of a sequence of $n$ observations from a moving average of order 1. Solutions are first given in terms of repeated integrals and then for the case where the underlying independent random variables are discrete. When the correlation is positive, $$ P(M_n \max^n_{i=1} X_i \leq x) = \sum_{j=1}^\infty β_{jx} ν_{jx}^{n} \approx B_{x} r{1x}^{n} $$ where…
▽ More
We give the distribution of $M_n$, the maximum of a sequence of $n$ observations from a moving average of order 1. Solutions are first given in terms of repeated integrals and then for the case where the underlying independent random variables are discrete. When the correlation is positive, $$ P(M_n \max^n_{i=1} X_i \leq x) = \sum_{j=1}^\infty β_{jx} ν_{jx}^{n} \approx B_{x} r{1x}^{n} $$ where $\{ν_{jx}\}$ are the eigenvalues of a certain matrix, $r_{1x}$ is the maximum magnitude of the eigenvalues, and $I$ depends on the number of possible values of the underlying random variables. The eigenvalues do not depend on $x$ only on its range.
△ Less
Submitted 6 April, 2009; v1 submitted 4 February, 2008;
originally announced February 2008.
-
The distribution of the maximum of a first order moving average: the continuous case
Authors:
Christopher S. Withers,
Saralees Nadarajah
Abstract:
We give the distribution of $M_n$, the maximum of a sequence of $n$ observations from a moving average of order 1. Solutions are first given in terms of repeated integrals and then for the case where the underlying independent random variables have an absolutely continuous density. When the correlation is positive,…
▽ More
We give the distribution of $M_n$, the maximum of a sequence of $n$ observations from a moving average of order 1. Solutions are first given in terms of repeated integrals and then for the case where the underlying independent random variables have an absolutely continuous density. When the correlation is positive, $$ P(M_n %\max^n_{i=1} X_i \leq x) =\ \sum_{j=1}^\infty β_{jx} ν_{jx}^{n} \approx B_{x} ν_{1x}^{n} $$ where %$\{X_i\}$ is a moving average of order 1 with positive correlation, and $\{ν_{jx}\}$ are the eigenvalues (singular values) of a Fredholm kernel and $ν_{1x}$ is the eigenvalue of maximum magnitude. A similar result is given when the correlation is negative. The result is analogous to large deviations expansions for estimates, since the maximum need not be standardized to have a limit. % there are more terms, and $$P(M_n <x) \approx B'_{x}\ (1+ν_{1x})^n.$$
For the continuous case the integral equations for the left and right eigenfunctions are converted to first order linear differential equations. The eigenvalues satisfy an equation of the form $$\sum_{i=1}^\infty w_i(λ-θ_i)^{-1}=λ-θ_0$$ for certain known weights $\{w_i\}$ and eigenvalues $\{θ_i\}$ of a given matrix. This can be solved by truncating the sum to an increasing number of terms.
△ Less
Submitted 6 September, 2009; v1 submitted 4 February, 2008;
originally announced February 2008.
