-
Fitting Laplacian Regularized Stratified Gaussian Models
Authors:
Jonathan Tuck,
Stephen Boyd
Abstract:
We consider the problem of jointly estimating multiple related zero-mean Gaussian distributions from data. We propose to jointly estimate these covariance matrices using Laplacian regularized stratified model fitting, which includes loss and regularization terms for each covariance matrix, and also a term that encourages the different covariances matrices to be close. This method `borrows strength…
▽ More
We consider the problem of jointly estimating multiple related zero-mean Gaussian distributions from data. We propose to jointly estimate these covariance matrices using Laplacian regularized stratified model fitting, which includes loss and regularization terms for each covariance matrix, and also a term that encourages the different covariances matrices to be close. This method `borrows strength' from the neighboring covariances, to improve its estimate. With well chosen hyper-parameters, such models can perform very well, especially in the low data regime. We propose a distributed method that scales to large problems, and illustrate the efficacy of the method with examples in finance, radar signal processing, and weather forecasting.
△ Less
Submitted 22 May, 2020; v1 submitted 4 May, 2020;
originally announced May 2020.
-
Convex Optimization Over Risk-Neutral Probabilities
Authors:
Shane Barratt,
Jonathan Tuck,
Stephen Boyd
Abstract:
We consider a collection of derivatives that depend on the price of an underlying asset at expiration or maturity. The absence of arbitrage is equivalent to the existence of a risk-neutral probability distribution on the price; in particular, any risk neutral distribution can be interpreted as a certificate establishing that no arbitrage exists. We are interested in the case when there are multipl…
▽ More
We consider a collection of derivatives that depend on the price of an underlying asset at expiration or maturity. The absence of arbitrage is equivalent to the existence of a risk-neutral probability distribution on the price; in particular, any risk neutral distribution can be interpreted as a certificate establishing that no arbitrage exists. We are interested in the case when there are multiple risk-neutral probabilities. We describe a number of convex optimization problems over the convex set of risk neutral price probabilities. These include computation of bounds on the cumulative distribution, VaR, CVaR, and other quantities, over the set of risk-neutral probabilities. After discretizing the underlying price, these problems become finite dimensional convex or quasiconvex optimization problems, and therefore are tractable. We illustrate our approach using real options and futures pricing data for the S&P 500 index and Bitcoin.
△ Less
Submitted 5 March, 2020;
originally announced March 2020.
-
Eigen-Stratified Models
Authors:
Jonathan Tuck,
Stephen Boyd
Abstract:
Stratified models depend in an arbitrary way on a selected categorical feature that takes $K$ values, and depend linearly on the other $n$ features. Laplacian regularization with respect to a graph on the feature values can greatly improve the performance of a stratified model, especially in the low-data regime. A significant issue with Laplacian-regularized stratified models is that the model is…
▽ More
Stratified models depend in an arbitrary way on a selected categorical feature that takes $K$ values, and depend linearly on the other $n$ features. Laplacian regularization with respect to a graph on the feature values can greatly improve the performance of a stratified model, especially in the low-data regime. A significant issue with Laplacian-regularized stratified models is that the model is $K$ times the size of the base model, which can be quite large.
We address this issue by formulating eigen-stratifed models, which are stratified models with an additional constraint that the model parameters are linear combinations of some modest number $m$ of bottom eigenvectors of the graph Laplacian, i.e., those associated with the $m$ smallest eigenvalues. With eigen-stratified models, we only need to store the $m$ bottom eigenvectors and the corresponding coefficients as the stratified model parameters. This leads to a reduction, sometimes large, of model size when $m \leq n$ and $m \ll K$. In some cases, the additional regularization implicit in eigen-stratified models can improve out-of-sample performance over standard Laplacian regularized stratified models.
△ Less
Submitted 27 January, 2020;
originally announced January 2020.
-
A Distributed Method for Fitting Laplacian Regularized Stratified Models
Authors:
Jonathan Tuck,
Shane Barratt,
Stephen Boyd
Abstract:
Stratified models are models that depend in an arbitrary way on a set of selected categorical features, and depend linearly on the other features. In a basic and traditional formulation a separate model is fit for each value of the categorical feature, using only the data that has the specific categorical value. To this formulation we add Laplacian regularization, which encourages the model parame…
▽ More
Stratified models are models that depend in an arbitrary way on a set of selected categorical features, and depend linearly on the other features. In a basic and traditional formulation a separate model is fit for each value of the categorical feature, using only the data that has the specific categorical value. To this formulation we add Laplacian regularization, which encourages the model parameters for neighboring categorical values to be similar. Laplacian regularization allows us to specify one or more weighted graphs on the stratification feature values. For example, stratifying over the days of the week, we can specify that the Sunday model parameter should be close to the Saturday and Monday model parameters. The regularization improves the performance of the model over the traditional stratified model, since the model for each value of the categorical `borrows strength' from its neighbors. In particular, it produces a model even for categorical values that did not appear in the training data set.
We propose an efficient distributed method for fitting stratified models, based on the alternating direction method of multipliers (ADMM). When the fitting loss functions are convex, the stratified model fitting problem is convex, and our method computes the global minimizer of the loss plus regularization; in other cases it computes a local minimizer. The method is very efficient, and naturally scales to large data sets or numbers of stratified feature values. We illustrate our method with a variety of examples.
△ Less
Submitted 10 November, 2019; v1 submitted 26 April, 2019;
originally announced April 2019.
