-
Lasso and elastic nets by orthants
Authors:
Hugo Maruri-Aguilar
Abstract:
We propose a new method for computing the lasso path, using the fact that the Manhattan norm of the coefficient vector is linear over every orthant of the parameter space. We use simple calculus and present an algorithm in which the lasso path is series of orthant moves. Our proposal gives the same results as standard literature, with the advantage of neat interpretation of results and explicit la…
▽ More
We propose a new method for computing the lasso path, using the fact that the Manhattan norm of the coefficient vector is linear over every orthant of the parameter space. We use simple calculus and present an algorithm in which the lasso path is series of orthant moves. Our proposal gives the same results as standard literature, with the advantage of neat interpretation of results and explicit lasso formulæ. We extend this proposal to elastic nets and obtain explicit, exact formulæ for the elastic net path, and with a simple change, our lasso algorithm can be used for elastic nets. We present computational examples and provide simple R prototype code.
△ Less
Submitted 19 July, 2023;
originally announced July 2023.
-
Lasso for hierarchical polynomial models
Authors:
Hugo Maruri-Aguilar,
Simon Lunagomez
Abstract:
In a polynomial regression model, the divisibility conditions implicit in polynomial hierarchy give way to a natural construction of constraints for the model parameters. We use this principle to derive versions of strong and weak hierarchy and to extend existing work in the literature, which at the moment is only concerned with models of degree two. We discuss how to estimate parameters in lasso…
▽ More
In a polynomial regression model, the divisibility conditions implicit in polynomial hierarchy give way to a natural construction of constraints for the model parameters. We use this principle to derive versions of strong and weak hierarchy and to extend existing work in the literature, which at the moment is only concerned with models of degree two. We discuss how to estimate parameters in lasso using standard quadratic programming techniques and apply our proposal to both simulated data and examples from the literature. The proposed methodology compares favorably with existing techniques in terms of low validation error and model size.
△ Less
Submitted 21 January, 2020;
originally announced January 2020.
-
The algebraic method in experimental design
Authors:
Hugo Maruri-Aguilar,
Henry P. Wynn
Abstract:
The algebraic method provides useful techniques to identify models in designs and to understand aliasing of polynomial models. The present note surveys the topic of Gröbner bases in experimental design and then describes the notion of confounding and the algebraic fan of a design. The ideas are illustrated with a variety of design examples ranging from Latin squares to screening designs.
The algebraic method provides useful techniques to identify models in designs and to understand aliasing of polynomial models. The present note surveys the topic of Gröbner bases in experimental design and then describes the notion of confounding and the algebraic fan of a design. The ideas are illustrated with a variety of design examples ranging from Latin squares to screening designs.
△ Less
Submitted 12 July, 2012;
originally announced July 2012.
-
Smooth supersaturated models
Authors:
Ron A. Bates,
Hugo Maruri-Aguilar,
Henry P. Wynn
Abstract:
In areas such as kernel smoothing and non-parametric regression there is emphasis on smooth interpolation and smooth statistical models. Splines are known to have optimal smoothness properties in one and higher dimensions. It is shown, with special attention to polynomial models, that smooth interpolators can be constructed by first extending the monomial basis and then minimising a measure of s…
▽ More
In areas such as kernel smoothing and non-parametric regression there is emphasis on smooth interpolation and smooth statistical models. Splines are known to have optimal smoothness properties in one and higher dimensions. It is shown, with special attention to polynomial models, that smooth interpolators can be constructed by first extending the monomial basis and then minimising a measure of smoothness with respect to the free parameters in the extended basis. Algebraic methods are a help in choosing the extended basis which can also be found as a saturated basis for an extended experimental design with dummy design points. One can get arbitrarily close to optimal smoothing for any dimension and over any region, giving a simple alternative models of spline type. The relationship to splines is shown in one and two dimensions. A case study is given which includes benchmarking against kriging methods.
△ Less
Submitted 26 September, 2008;
originally announced September 2008.
-
Minimal average degree aberration and the state polytope for experimental designs
Authors:
Yael Berstein,
Hugo Maruri-Aguilar,
Shmuel Onn,
Eva Riccomagno,
Henry Wynn
Abstract:
For a particular experimental design, there is interest in finding which polynomial models can be identified in the usual regression set up. The algebraic methods based on Groebner bases provide a systematic way of doing this. The algebraic method does not in general produce all estimable models but it can be shown that it yields models which have minimal average degree in a well-defined sense a…
▽ More
For a particular experimental design, there is interest in finding which polynomial models can be identified in the usual regression set up. The algebraic methods based on Groebner bases provide a systematic way of doing this. The algebraic method does not in general produce all estimable models but it can be shown that it yields models which have minimal average degree in a well-defined sense and in both a weighted and unweighted version. This provides an alternative measure to that based on "aberration" and moreover is applicable to any experimental design. A simple algorithm is given and bounds are derived for the criteria, which may be used to give asymptotic Nyquist-like estimability rates as model and sample sizes increase.
△ Less
Submitted 22 August, 2008;
originally announced August 2008.
-
Nonlinear Matroid Optimization and Experimental Design
Authors:
Yael Berstein,
Jon Lee,
Hugo Maruri-Aguilar,
Shmuel Onn,
Eva Riccomagno,
Robert Weismantel,
Henry Wynn
Abstract:
We study the problem of optimizing nonlinear objective functions over matroids presented by oracles or explicitly. Such functions can be interpreted as the balancing of multi-criteria optimization. We provide a combinatorial polynomial time algorithm for arbitrary oracle-presented matroids, that makes repeated use of matroid intersection, and an algebraic algorithm for vectorial matroids.
Our…
▽ More
We study the problem of optimizing nonlinear objective functions over matroids presented by oracles or explicitly. Such functions can be interpreted as the balancing of multi-criteria optimization. We provide a combinatorial polynomial time algorithm for arbitrary oracle-presented matroids, that makes repeated use of matroid intersection, and an algebraic algorithm for vectorial matroids.
Our work is partly motivated by applications to minimum-aberration model-fitting in experimental design in statistics, which we discuss and demonstrate in detail.
△ Less
Submitted 31 July, 2007;
originally announced July 2007.
