Showing 1–22 of 22 results for author: Hosseini, R

Designing a Bayesian adaptive clinical trial to evaluate novel mechanical ventilation strategies in acute respiratory failure using Integrated Nested Laplace Approximations

Authors: Reyhaneh Hosseini, Ziming Chen, Ewan Goligher, Eddy Fan, Niall D. Ferguson, Michael O. Harhay, Sarina Sahetya, Martin Urner, Christopher J. Yarnell, Anna Heath

Abstract: Background: We aimed to design a Bayesian adaption trial through extensive simulations to determine values for key design parameters, demonstrate error rates, and establish the expected sample size. The complexity of the proposed outcome and analysis meant that Markov Chain Monte Carlo methods were required, resulting in an infeasible computational burden. Thus, we leveraged the Integrated Nested… ▽ More Background: We aimed to design a Bayesian adaption trial through extensive simulations to determine values for key design parameters, demonstrate error rates, and establish the expected sample size. The complexity of the proposed outcome and analysis meant that Markov Chain Monte Carlo methods were required, resulting in an infeasible computational burden. Thus, we leveraged the Integrated Nested Laplace Approximations (INLA) algorithm, a fast approximation method, to ensure the feasibility of these simulations. Methods: We simulated Bayesian adaptive two-arm superiority trials that stratified participants into two disease severity states. The outcome was analyzed with proportional odds logistic regression. Trials were stopped for superiority or futility, separately for each state. We calculated the type I error and power across 64 scenarios that varied the stopping thresholds and the minimum sample size before commencing adaptive analyses. We incorporated dynamic borrowing and used INLA to compute the posterior distributions at each adaptive analysis. Designs that maintained a type I error below 5%, a power above 80%, and a feasible mean sample size were then evaluated across 22 scenarios that varied the odds ratios for the two severity states. Results: Power generally increased as the initial sample size and the threshold for declaring futility increased. Two designs were selected for further analysis. In the comprehensive simulations, the one design had a higher chance of reaching a trial conclusion before the maximum sample size and higher probability of declaring superiority when appropriate without a substantial increase in sample size for the more realistic scenarios and was selected as the trial design. Conclusions: We designed a Bayesian adaptive trial to evaluate novel strategies for ventilation using the INLA algorithm to and optimize the trial design through simulation. △ Less

Submitted 31 March, 2023; originally announced March 2023.

Greykite: Deploying Flexible Forecasting at Scale at LinkedIn

Authors: Reza Hosseini, Albert Chen, Kaixu Yang, Sayan Patra, Yi Su, Saad Eddin Al Orjany, Sishi Tang, Parvez Ahammad

Abstract: Forecasts help businesses allocate resources and achieve objectives. At LinkedIn, product owners use forecasts to set business targets, track outlook, and monitor health. Engineers use forecasts to efficiently provision hardware. Developing a forecasting solution to meet these needs requires accurate and interpretable forecasts on diverse time series with sub-hourly to quarterly frequencies. We pr… ▽ More Forecasts help businesses allocate resources and achieve objectives. At LinkedIn, product owners use forecasts to set business targets, track outlook, and monitor health. Engineers use forecasts to efficiently provision hardware. Developing a forecasting solution to meet these needs requires accurate and interpretable forecasts on diverse time series with sub-hourly to quarterly frequencies. We present Greykite, an open-source Python library for forecasting that has been deployed on over twenty use cases at LinkedIn. Its flagship algorithm, Silverkite, provides interpretable, fast, and highly flexible univariate forecasts that capture effects such as time-varying growth and seasonality, autocorrelation, holidays, and regressors. The library enables self-serve accuracy and trust by facilitating data exploration, model configuration, execution, and interpretation. Our benchmark results show excellent out-of-the-box speed and accuracy on datasets from a variety of domains. Over the past two years, Greykite forecasts have been trusted by Finance, Engineering, and Product teams for resource planning and allocation, target setting and progress tracking, anomaly detection and root cause analysis. We expect Greykite to be useful to forecast practitioners with similar applications who need accurate, interpretable forecasts that capture complex dynamics common to time series related to human activity. △ Less

Submitted 15 July, 2022; originally announced July 2022.

Comments: In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD '22), August 14-18, 2022, Washington, DC, USA. ACM, New York, NY, USA, 11 pages

ACM Class: G.3

Joint Manifold Learning and Density Estimation Using Normalizing Flows

Authors: Seyedeh Fatemeh Razavi, Mohammad Mahdi Mehmanchi, Reshad Hosseini, Mostafa Tavassolipour

Abstract: Based on the manifold hypothesis, real-world data often lie on a low-dimensional manifold, while normalizing flows as a likelihood-based generative model are incapable of finding this manifold due to their structural constraints. So, one interesting question arises: $\textit{"Can we find sub-manifold(s) of data in normalizing flows and estimate the density of the data on the sub-manifold(s)?"}$. I… ▽ More Based on the manifold hypothesis, real-world data often lie on a low-dimensional manifold, while normalizing flows as a likelihood-based generative model are incapable of finding this manifold due to their structural constraints. So, one interesting question arises: $\textit{"Can we find sub-manifold(s) of data in normalizing flows and estimate the density of the data on the sub-manifold(s)?"}$. In this paper, we introduce two approaches, namely per-pixel penalized log-likelihood and hierarchical training, to answer the mentioned question. We propose a single-step method for joint manifold learning and density estimation by disentangling the transformed space obtained by normalizing flows to manifold and off-manifold parts. This is done by a per-pixel penalized likelihood function for learning a sub-manifold of the data. Normalizing flows assume the transformed data is Gaussianizationed, but this imposed assumption is not necessarily true, especially in high dimensions. To tackle this problem, a hierarchical training approach is employed to improve the density estimation on the sub-manifold. The results validate the superiority of the proposed methods in simultaneous manifold learning and density estimation using normalizing flows in terms of generated image quality and likelihood. △ Less

Submitted 7 June, 2022; originally announced June 2022.

Vector Transport Free Riemannian LBFGS for Optimization on Symmetric Positive Definite Matrix Manifolds

Authors: Reza Godaz, Benyamin Ghojogh, Reshad Hosseini, Reza Monsefi, Fakhri Karray, Mark Crowley

Abstract: This work concentrates on optimization on Riemannian manifolds. The Limited-memory Broyden-Fletcher-Goldfarb-Shanno (LBFGS) algorithm is a commonly used quasi-Newton method for numerical optimization in Euclidean spaces. Riemannian LBFGS (RLBFGS) is an extension of this method to Riemannian manifolds. RLBFGS involves computationally expensive vector transports as well as unfolding recursions using… ▽ More This work concentrates on optimization on Riemannian manifolds. The Limited-memory Broyden-Fletcher-Goldfarb-Shanno (LBFGS) algorithm is a commonly used quasi-Newton method for numerical optimization in Euclidean spaces. Riemannian LBFGS (RLBFGS) is an extension of this method to Riemannian manifolds. RLBFGS involves computationally expensive vector transports as well as unfolding recursions using adjoint vector transports. In this article, we propose two mappings in the tangent space using the inverse second root and Cholesky decomposition. These mappings make both vector transport and adjoint vector transport identity and therefore isometric. Identity vector transport makes RLBFGS less computationally expensive and its isometry is also very useful in convergence analysis of RLBFGS. Moreover, under the proposed mappings, the Riemannian metric reduces to Euclidean inner product, which is much less computationally expensive. We focus on the Symmetric Positive Definite (SPD) manifolds which are beneficial in various fields such as data science and statistics. This work opens a research opportunity for extension of the proposed mappings to other well-known manifolds. △ Less

Submitted 3 October, 2021; v1 submitted 24 August, 2021; originally announced August 2021.

Comments: Published in the 13th Asian Conference on Machine Learning (ACML) 2021. The first two authors contributed equally to this work

Report number: https://proceedings.mlr.press/v157/godaz21a.html

Journal ref: Proceedings of The 13th Asian Conference on Machine Learning, PMLR, vol. 157, pp. 1-16, 2021

A flexible forecasting model for production systems

Authors: Reza Hosseini, Kaixu Yang, Albert Chen, Sayan Patra

Abstract: This paper discusses desirable properties of forecasting models in production systems. It then develops a family of models which are designed to satisfy these properties: highly customizable to capture complex patterns; accommodates a large variety of objectives; has interpretable components; produces robust results; has automatic changepoint detection for trend and seasonality; and runs fast -- m… ▽ More This paper discusses desirable properties of forecasting models in production systems. It then develops a family of models which are designed to satisfy these properties: highly customizable to capture complex patterns; accommodates a large variety of objectives; has interpretable components; produces robust results; has automatic changepoint detection for trend and seasonality; and runs fast -- making it a good choice for reliable and scalable production systems. The model allows for seasonality at various time scales, events/holidays, and change points in trend and seasonality. The volatility is fitted separately to maintain flexibility and speed and is allowed to be a function of specified features. △ Less

Submitted 3 May, 2021; originally announced May 2021.

Accurate and fast matrix factorization for low-rank learning

Authors: Reza Godaz, Reza Monsefi, Faezeh Toutounian, Reshad Hosseini

Abstract: In this paper, we tackle two important problems in low-rank learning, which are partial singular value decomposition and numerical rank estimation of huge matrices. By using the concepts of Krylov subspaces such as Golub-Kahan bidiagonalization (GK-bidiagonalization) as well as Ritz vectors, we propose two methods for solving these problems in a fast and accurate way. Our experiments show the adva… ▽ More In this paper, we tackle two important problems in low-rank learning, which are partial singular value decomposition and numerical rank estimation of huge matrices. By using the concepts of Krylov subspaces such as Golub-Kahan bidiagonalization (GK-bidiagonalization) as well as Ritz vectors, we propose two methods for solving these problems in a fast and accurate way. Our experiments show the advantages of the proposed methods compared to the traditional and randomized singular value decomposition methods. The proposed methods are appropriate for applications involving huge matrices where the accuracy of the desired singular values and also all of their corresponding singular vectors are essential. As a real application, we evaluate the performance of our methods on the problem of Riemannian similarity learning between two various image datasets of MNIST and USPS. △ Less

Submitted 4 September, 2021; v1 submitted 21 April, 2021; originally announced April 2021.

Model selection for count timeseries with applications in forecasting number of trips in bike-sharing systems and its volatility

Authors: Alireza Hosseini, Reza Hosseini

Abstract: Forecasting the number of trips in bike-sharing systems and its volatility over time is crucial for planning and optimizing such systems. This paper develops timeseries models to forecast hourly count timeseries data, and estimate its volatility. Such models need to take into account the complex patterns over various temporal scales including hourly, daily, weekly and annual as well as the tempora… ▽ More Forecasting the number of trips in bike-sharing systems and its volatility over time is crucial for planning and optimizing such systems. This paper develops timeseries models to forecast hourly count timeseries data, and estimate its volatility. Such models need to take into account the complex patterns over various temporal scales including hourly, daily, weekly and annual as well as the temporal correlation. To capture this complex structure, a large number of parameters are needed. Here a structural model selection approach is utilized to choose the parameters. This method explores the parameter space for a group of covariates at each step. These groups of covariate are constructed to represent a particular structure in the model. The statistical models utilized are extensions of Generalized Linear Models to timeseries data. One challenge in using such models is the explosive behavior of the simulated values. To address this issue, we develop a technique which relies on damping the simulated value, if it falls outside of an admissible interval. The admissible interval is defined using measures of variability of the left and right tails. A new definition of outliers is proposed based on these variability measures. This new definition is shown to be useful in the context of asymmetric distributions. △ Less

Submitted 16 November, 2020; originally announced November 2020.

FRMDN: Flow-based Recurrent Mixture Density Network

Authors: Seyedeh Fatemeh Razavi, Reshad Hosseini, Tina Behzad

Abstract: The class of recurrent mixture density networks is an important class of probabilistic models used extensively in sequence modeling and sequence-to-sequence mapping applications. In this class of models, the density of a target sequence in each time-step is modeled by a Gaussian mixture model with the parameters given by a recurrent neural network. In this paper, we generalize recurrent mixture de… ▽ More The class of recurrent mixture density networks is an important class of probabilistic models used extensively in sequence modeling and sequence-to-sequence mapping applications. In this class of models, the density of a target sequence in each time-step is modeled by a Gaussian mixture model with the parameters given by a recurrent neural network. In this paper, we generalize recurrent mixture density networks by defining a Gaussian mixture model on a non-linearly transformed target sequence in each time-step. The non-linearly transformed space is created by normalizing flow. We observed that this model significantly improves the fit to image sequences measured by the log-likelihood. We also applied the proposed model on some speech and image data, and observed that the model has significant modeling power outperforming other state-of-the-art methods in terms of the log-likelihood. △ Less

Submitted 20 April, 2023; v1 submitted 5 August, 2020; originally announced August 2020.

Pathway-Activity Likelihood Analysis and Metabolite Annotation for Untargeted Metabolomics using Probabilistic Modeling

Authors: Ramtin Hosseini, Neda Hassanpour, Li-Ping Liu, Soha Hassoun

Abstract: Motivation: Untargeted metabolomics comprehensively characterizes small molecules and elucidates activities of biochemical pathways within a biological sample. Despite computational advances, interpreting collected measurements and determining their biological role remains a challenge. Results: To interpret measurements, we present an inference-based approach, termed Probabilistic modeling for Unt… ▽ More Motivation: Untargeted metabolomics comprehensively characterizes small molecules and elucidates activities of biochemical pathways within a biological sample. Despite computational advances, interpreting collected measurements and determining their biological role remains a challenge. Results: To interpret measurements, we present an inference-based approach, termed Probabilistic modeling for Untargeted Metabolomics Analysis (PUMA). Our approach captures measurements and known information about the sample under study in a generative model and uses stochastic sampling to compute posterior probability distributions. PUMA predicts the likelihood of pathways being active, and then derives a probabilistic annotation, which assigns chemical identities to the measurements. PUMA is validated on synthetic datasets. When applied to test cases, the resulting pathway activities are biologically meaningful and distinctly different from those obtained using statistical pathway enrichment techniques. Annotation results are in agreement to those obtained using other tools that utilize additional information in the form of spectral signatures. Importantly, PUMA annotates many additional measurements. △ Less

Submitted 9 March, 2020; v1 submitted 11 December, 2019; originally announced December 2019.

Comments: For more details, please visit my homepage at: https://www.eecs.tufts.edu/~ramtin/

Active Transfer Learning for Persian Offline Signature Verification

Authors: Taraneh Younesian, Saeed Masoudnia, Reshad Hosseini, Babak N. Araabi

Abstract: Offline Signature Verification (OSV) remains a challenging pattern recognition task, especially in the presence of skilled forgeries that are not available during the training. This challenge is aggravated when there are small labeled training data available but with large intra-personal variations. In this study, we address this issue by employing an active learning approach, which selects the mo… ▽ More Offline Signature Verification (OSV) remains a challenging pattern recognition task, especially in the presence of skilled forgeries that are not available during the training. This challenge is aggravated when there are small labeled training data available but with large intra-personal variations. In this study, we address this issue by employing an active learning approach, which selects the most informative instances to label and therefore reduces the human labeling effort significantly. Our proposed OSV includes three steps: feature learning, active learning, and final verification. We benefit from transfer learning using a pre-trained CNN for feature learning. We also propose SVM-based active learning for each user to separate his genuine signatures from the random forgeries. We finally used the SVMs to verify the authenticity of the questioned signature. We examined our proposed active transfer learning method on UTSig: A Persian offline signature dataset. We achieved near 13% improvement compared to the random selection of instances. Our results also showed 1% improvement over the state-of-the-art method in which a fully supervised setting with five more labeled instances per user was used. △ Less

Submitted 28 February, 2019; originally announced March 2019.

Journal ref: 2019 4th International Conference on Pattern Recognition and Image Analysis (IPRIA)

Deep-RBF Networks Revisited: Robust Classification with Rejection

Authors: Pourya Habib Zadeh, Reshad Hosseini, Suvrit Sra

Abstract: One of the main drawbacks of deep neural networks, like many other classifiers, is their vulnerability to adversarial attacks. An important reason for their vulnerability is assigning high confidence to regions with few or even no feature points. By feature points, we mean a nonlinear transformation of the input space extracting a meaningful representation of the input data. On the other hand, dee… ▽ More One of the main drawbacks of deep neural networks, like many other classifiers, is their vulnerability to adversarial attacks. An important reason for their vulnerability is assigning high confidence to regions with few or even no feature points. By feature points, we mean a nonlinear transformation of the input space extracting a meaningful representation of the input data. On the other hand, deep-RBF networks assign high confidence only to the regions containing enough feature points, but they have been discounted due to the widely-held belief that they have the vanishing gradient problem. In this paper, we revisit the deep-RBF networks by first giving a general formulation for them, and then proposing a family of cost functions thereof inspired by metric learning. In the proposed deep-RBF learning algorithm, the vanishing gradient problem does not occur. We make these networks robust to adversarial attack by adding the reject option to their output layer. Through several experiments on the MNIST dataset, we demonstrate that our proposed method not only achieves significant classification accuracy but is also very resistant to various adversarial attacks. △ Less

Submitted 7 December, 2018; originally announced December 2018.

Exploiting generalization in the subspaces for faster model-based learning

Authors: Maryam Hashemzadeh, Reshad Hosseini, Majid Nili Ahmadabadi

Abstract: Due to the lack of enough generalization in the state-space, common methods in Reinforcement Learning (RL) suffer from slow learning speed especially in the early learning trials. This paper introduces a model-based method in discrete state-spaces for increasing learning speed in terms of required experience (but not required computational time) by exploiting generalization in the experiences of t… ▽ More Due to the lack of enough generalization in the state-space, common methods in Reinforcement Learning (RL) suffer from slow learning speed especially in the early learning trials. This paper introduces a model-based method in discrete state-spaces for increasing learning speed in terms of required experience (but not required computational time) by exploiting generalization in the experiences of the subspaces. A subspace is formed by choosing a subset of features in the original state representation (full-space). Generalization and faster learning in a subspace are due to many-to-one mapping of experiences from the full-space to each state in the subspace. Nevertheless, due to inherent perceptual aliasing in the subspaces, the policy suggested by each subspace does not generally converge to the optimal policy. Our approach, called Model Based Learning with Subspaces (MoBLeS), calculates confidence intervals of the estimated Q-values in the full-space and in the subspaces. These confidence intervals are used in the decision making, such that the agent benefits the most from the possible generalization while avoiding from detriment of the perceptual aliasing in the subspaces. Convergence of MoBLeS to the optimal policy is theoretically investigated. Additionally, we show through several experiments that MoBLeS improves the learning speed in the early trials. △ Less

Submitted 25 October, 2017; v1 submitted 22 October, 2017; originally announced October 2017.

An Alternative to EM for Gaussian Mixture Models: Batch and Stochastic Riemannian Optimization

Authors: Reshad Hosseini, Suvrit Sra

Abstract: We consider maximum likelihood estimation for Gaussian Mixture Models (Gmms). This task is almost invariably solved (in theory and practice) via the Expectation Maximization (EM) algorithm. EM owes its success to various factors, of which is its ability to fulfill positive definiteness constraints in closed form is of key importance. We propose an alternative to EM by appealing to the rich Riemann… ▽ More We consider maximum likelihood estimation for Gaussian Mixture Models (Gmms). This task is almost invariably solved (in theory and practice) via the Expectation Maximization (EM) algorithm. EM owes its success to various factors, of which is its ability to fulfill positive definiteness constraints in closed form is of key importance. We propose an alternative to EM by appealing to the rich Riemannian geometry of positive definite matrices, using which we cast Gmm parameter estimation as a Riemannian optimization problem. Surprisingly, such an out-of-the-box Riemannian formulation completely fails and proves much inferior to EM. This motivates us to take a closer look at the problem geometry, and derive a better formulation that is much more amenable to Riemannian optimization. We then develop (Riemannian) batch and stochastic gradient algorithms that outperform EM, often substantially. We provide a non-asymptotic convergence analysis for our stochastic method, which is also the first (to our knowledge) such global analysis for Riemannian stochastic gradient. Numerous empirical results are included to demonstrate the effectiveness of our methods. △ Less

Submitted 10 June, 2017; originally announced June 2017.

Comments: 21 pages, 6 figures

Geometric Mean Metric Learning

Authors: Pourya Habib Zadeh, Reshad Hosseini, Suvrit Sra

Abstract: We revisit the task of learning a Euclidean metric from data. We approach this problem from first principles and formulate it as a surprisingly simple optimization problem. Indeed, our formulation even admits a closed form solution. This solution possesses several very attractive properties: (i) an innate geometric appeal through the Riemannian geometry of positive definite matrices; (ii) ease of… ▽ More We revisit the task of learning a Euclidean metric from data. We approach this problem from first principles and formulate it as a surprisingly simple optimization problem. Indeed, our formulation even admits a closed form solution. This solution possesses several very attractive properties: (i) an innate geometric appeal through the Riemannian geometry of positive definite matrices; (ii) ease of interpretability; and (iii) computational speed several orders of magnitude faster than the widely used LMNN and ITML methods. Furthermore, on standard benchmark datasets, our closed-form solution consistently attains higher classification accuracy. △ Less

Submitted 18 July, 2016; originally announced July 2016.

Comments: 7 pages, 4 figures

MixEst: An Estimation Toolbox for Mixture Models

Authors: Reshad Hosseini, Mohamadreza Mash'al

Abstract: Mixture models are powerful statistical models used in many applications ranging from density estimation to clustering and classification. When dealing with mixture models, there are many issues that the experimenter should be aware of and needs to solve. The MixEst toolbox is a powerful and user-friendly package for MATLAB that implements several state-of-the-art approaches to address these probl… ▽ More Mixture models are powerful statistical models used in many applications ranging from density estimation to clustering and classification. When dealing with mixture models, there are many issues that the experimenter should be aware of and needs to solve. The MixEst toolbox is a powerful and user-friendly package for MATLAB that implements several state-of-the-art approaches to address these problems. Additionally, MixEst gives the possibility of using manifold optimization for fitting the density model, a feature specific to this toolbox. MixEst simplifies using and integration of mixture models in statistical models and applications. For developing mixture models of new densities, the user just needs to provide a few functions for that statistical distribution and the toolbox takes care of all the issues regarding mixture models. MixEst is available at visionlab.ut.ac.ir/mixest and is fully documented and is licensed under GPL. △ Less

Submitted 22 July, 2015; originally announced July 2015.

Comments: 5 pages

Manifold Optimization for Gaussian Mixture Models

Authors: Reshad Hosseini, Suvrit Sra

Abstract: We take a new look at parameter estimation for Gaussian Mixture Models (GMMs). In particular, we propose using \emph{Riemannian manifold optimization} as a powerful counterpart to Expectation Maximization (EM). An out-of-the-box invocation of manifold optimization, however, fails spectacularly: it converges to the same solution but vastly slower. Driven by intuition from manifold convexity, we the… ▽ More We take a new look at parameter estimation for Gaussian Mixture Models (GMMs). In particular, we propose using \emph{Riemannian manifold optimization} as a powerful counterpart to Expectation Maximization (EM). An out-of-the-box invocation of manifold optimization, however, fails spectacularly: it converges to the same solution but vastly slower. Driven by intuition from manifold convexity, we then propose a reparamerization that has remarkable empirical consequences. It makes manifold optimization not only match EM---a highly encouraging result in itself given the poor record nonlinear programming methods have had against EM so far---but also outperform EM in many practical settings, while displaying much less variability in running times. We further highlight the strengths of manifold optimization by developing a somewhat tuned manifold LBFGS method that proves even more competitive and reliable than existing manifold optimization tools. We hope that our results encourage a wider consideration of manifold optimization for parameter estimation problems. △ Less

Submitted 25 June, 2015; originally announced June 2015.

Comments: 19 pages

Inference and Mixture Modeling with the Elliptical Gamma Distribution

Authors: Reshad Hosseini, Suvrit Sra, Lucas Theis, Matthias Bethge

Abstract: We study modeling and inference with the Elliptical Gamma Distribution (EGD). We consider maximum likelihood (ML) estimation for EGD scatter matrices, a task for which we develop new fixed-point algorithms. Our algorithms are efficient and converge to global optima despite nonconvexity. Moreover, they turn out to be much faster than both a well-known iterative algorithm of Kent & Tyler (1991) and… ▽ More We study modeling and inference with the Elliptical Gamma Distribution (EGD). We consider maximum likelihood (ML) estimation for EGD scatter matrices, a task for which we develop new fixed-point algorithms. Our algorithms are efficient and converge to global optima despite nonconvexity. Moreover, they turn out to be much faster than both a well-known iterative algorithm of Kent & Tyler (1991) and sophisticated manifold optimization algorithms. Subsequently, we invoke our ML algorithms as subroutines for estimating parameters of a mixture of EGDs. We illustrate our methods by applying them to model natural image statistics---the proposed EGD mixture model yields the most parsimonious model among several competing approaches. △ Less

Submitted 20 December, 2015; v1 submitted 17 October, 2014; originally announced October 2014.

Comments: 23 pages, 11 figures

Journal ref: Computational Statistics & Data Analysis 2016, Vol. 101, 29-43

Utilizing wind in spatial covariance

Authors: Reza Hosseini

Abstract: This work develops a covariance function which allows for a stronger spatial correlation for pairs of points in the direction of a vector such as wind and weaker for pairs which are perpendicular to it. It derives a simple covariance function by stretching the space along the wind axes (upwind and across wind axes). It is shown that this covariance function is anisotropy in the original space and… ▽ More This work develops a covariance function which allows for a stronger spatial correlation for pairs of points in the direction of a vector such as wind and weaker for pairs which are perpendicular to it. It derives a simple covariance function by stretching the space along the wind axes (upwind and across wind axes). It is shown that this covariance function is anisotropy in the original space and the functions is explicitly calculated. △ Less

Submitted 13 August, 2014; originally announced August 2014.

A framework for fitting sparse data

Authors: Reza Hosseini, Akimichi Takemura, Kiros Berhane

Abstract: This paper develops a framework for fitting functions with domains in the Euclidean space, when data are sparse but a slow variation allows for a useful fit. We measure the variation by Lipschitz Bound (LB). Functions which admit smaller LB are considered to vary more slowly. Since most functions in practice are wiggly and do not admit a small LB, we extend this framework by approximating a wiggly… ▽ More This paper develops a framework for fitting functions with domains in the Euclidean space, when data are sparse but a slow variation allows for a useful fit. We measure the variation by Lipschitz Bound (LB). Functions which admit smaller LB are considered to vary more slowly. Since most functions in practice are wiggly and do not admit a small LB, we extend this framework by approximating a wiggly function, f, by ones which admit a smaller LB and do not deviate from f by more than a specified Bound Deviation (BD). In fact for any positive LB, one can find such a BD, thus defining a trade-off function (LB-BD function) between the variation measure (LB) and the deviation measure (BD). We show that the LB-BD function satisfies nice properties: it is non-increasing and convex. We also present a method to obtain it using convex optimization. For a function with given LB and BD, we find the optimal fit and present deterministic bounds for the prediction error of various methods. Given the LB-BD function, we discuss picking an appropriate LB-BD pair for fitting and calculating the prediction errors. The developed methods can naturally accommodate an extra assumption of periodicity to obtain better prediction errors. Finally we present the application of this framework to air pollution data with sparse observations over time. △ Less

Submitted 4 July, 2014; originally announced July 2014.

Mixtures of conditional Gaussian scale mixtures applied to multiscale image representations

Authors: Lucas Theis, Reshad Hosseini, Matthias Bethge

Abstract: We present a probabilistic model for natural images which is based on Gaussian scale mixtures and a simple multiscale representation. In contrast to the dominant approach to modeling whole images focusing on Markov random fields, we formulate our model in terms of a directed graphical model. We show that it is able to generate images with interesting higher-order correlations when trained on natur… ▽ More We present a probabilistic model for natural images which is based on Gaussian scale mixtures and a simple multiscale representation. In contrast to the dominant approach to modeling whole images focusing on Markov random fields, we formulate our model in terms of a directed graphical model. We show that it is able to generate images with interesting higher-order correlations when trained on natural images or samples from an occlusion based model. More importantly, the directed model enables us to perform a principled evaluation. While it is easy to generate visually appealing images, we demonstrate that our model also yields the best performance reported to date when evaluated with respect to the cross-entropy rate, a measure tightly linked to the average log-likelihood. △ Less

Submitted 20 September, 2011; originally announced September 2011.

Gravitational Lensing Accuracy Testing 2010 (GREAT10) Challenge Handbook

Authors: Thomas Kitching, Sreekumar Balan, Gary Bernstein, Matthias Bethge, Sarah Bridle, Frederic Courbin, Marc Gentile, Alan Heavens, Michael Hirsch, Reshad Hosseini, Alina Kiessling, Adam Amara, Donnacha Kirk, Konrad Kuijken, Rachel Mandelbaum, Baback Moghaddam, Guldariya Nurbaeva, Stephane Paulin-Henriksson, Anais Rassat, Jason Rhodes, Bernhard Schölkopf, John Shawe-Taylor, Mandeep Gill, Marina Shmakova, Andy Taylor , et al. (10 additional authors not shown)

Abstract: GRavitational lEnsing Accuracy Testing 2010 (GREAT10) is a public image analysis challenge aimed at the development of algorithms to analyze astronomical images. Specifically, the challenge is to measure varying image distortions in the presence of a variable convolution kernel, pixelization and noise. This is the second in a series of challenges set to the astronomy, computer science and statisti… ▽ More GRavitational lEnsing Accuracy Testing 2010 (GREAT10) is a public image analysis challenge aimed at the development of algorithms to analyze astronomical images. Specifically, the challenge is to measure varying image distortions in the presence of a variable convolution kernel, pixelization and noise. This is the second in a series of challenges set to the astronomy, computer science and statistics communities, providing a structured environment in which methods can be improved and tested in preparation for planned astronomical surveys. GREAT10 extends upon previous work by introducing variable fields into the challenge. The "Galaxy Challenge" involves the precise measurement of galaxy shape distortions, quantified locally by two parameters called shear, in the presence of a known convolution kernel. Crucially, the convolution kernel and the simulated gravitational lensing shape distortion both now vary as a function of position within the images, as is the case for real data. In addition, we introduce the "Star Challenge" that concerns the reconstruction of a variable convolution kernel, similar to that in a typical astronomical observation. This document details the GREAT10 Challenge for potential participants. Continually updated information is also available from http://www.greatchallenges.info. △ Less

Submitted 30 November, 2011; v1 submitted 3 September, 2010; originally announced September 2010.

Comments: Published in at http://dx.doi.org/10.1214/11-AOAS484 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AOAS-AOAS484

Journal ref: Annals of Applied Statistics 2011, Vol. 5, No. 3, 2231-2263

Approximating quantiles in very large datasets

Authors: Reza Hosseini

Abstract: Very large datasets are often encountered in climatology, either from a multiplicity of observations over time and space or outputs from deterministic models (sometimes in petabytes= 1 million gigabytes). Loading a large data vector and sorting it, is impossible sometimes due to memory limitations or computing power. We show that a proposed algorithm to approximating the median, "the median of the… ▽ More Very large datasets are often encountered in climatology, either from a multiplicity of observations over time and space or outputs from deterministic models (sometimes in petabytes= 1 million gigabytes). Loading a large data vector and sorting it, is impossible sometimes due to memory limitations or computing power. We show that a proposed algorithm to approximating the median, "the median of the median" performs poorly. Instead we develop an algorithm to approximate quantiles of very large datasets which works by partitioning the data or use existing partitions (possibly of non-equal size). We show the deterministic precision of this algorithm and how it can be adjusted to get customized precisions. △ Less

Submitted 6 July, 2010; originally announced July 2010.