-
Representing Scott sets in algebraic settings
Authors:
Alf Dolich,
Julia Knight,
Karen Lange,
David Marker
Abstract:
We prove that for every Scott set $S$ there are $S$-saturated real closed fields and models of Presburger arithmetic.
We prove that for every Scott set $S$ there are $S$-saturated real closed fields and models of Presburger arithmetic.
△ Less
Submitted 21 July, 2014;
originally announced July 2014.
-
$\mathcal{D}$-maximal sets
Authors:
Peter Cholak,
Peter Gerdes,
Karen Lange
Abstract:
Soare proved that the maximal sets form an orbit in $\mathcal{E}$. We consider here $\mathcal{D}$-maximal sets, generalizations of maximal sets introduced by Herrmann and Kummer. Some orbits of $\mathcal{D}$-maximal sets are well understood, e.g., hemimaximal sets, but many are not. The goal of this paper is to define new invariants on computably enumerable sets and to use them to give a complete…
▽ More
Soare proved that the maximal sets form an orbit in $\mathcal{E}$. We consider here $\mathcal{D}$-maximal sets, generalizations of maximal sets introduced by Herrmann and Kummer. Some orbits of $\mathcal{D}$-maximal sets are well understood, e.g., hemimaximal sets, but many are not. The goal of this paper is to define new invariants on computably enumerable sets and to use them to give a complete nontrivial classification of the $\mathcal{D}$-maximal sets. Although these invariants help us to better understand the $\mathcal{D}$-maximal sets, we use them to show that several classes of $\mathcal{D}$-maximal sets break into infinitely many orbits.
△ Less
Submitted 16 December, 2014; v1 submitted 6 January, 2014;
originally announced January 2014.
-
Chemical Bonding in Aqueous Ferrocyanide: Experimental and Theoretical X-ray Spectroscopic Study
Authors:
Nicholas Engel,
Sergey I. Bokarev,
Edlira Suljoti,
Raul Garcia-Diez,
Kathrin M. Lange,
Kaan Atak,
Ronny Golnak,
Alexander Kothe,
Marcus Dantz,
Oliver Kühn,
Emad F. Aziz
Abstract:
Resonant inelastic X-ray scattering (RIXS) and X-ray absorption (XA) experiments at the iron L- and nitrogen K-edge are combined with high-level first principles restricted active space self-consistent field (RASSCF) calculations for a systematic investigation of the nature of the chemical bond in potassium ferrocyanide in aqueous solution. The atom- and site-specific RIXS excitations allow for di…
▽ More
Resonant inelastic X-ray scattering (RIXS) and X-ray absorption (XA) experiments at the iron L- and nitrogen K-edge are combined with high-level first principles restricted active space self-consistent field (RASSCF) calculations for a systematic investigation of the nature of the chemical bond in potassium ferrocyanide in aqueous solution. The atom- and site-specific RIXS excitations allow for direct observation of ligand-to-metal (Fe L-edge) and metal-to-ligand (N K-edge) charge transfer bands and thereby evidence for strong σ-donation and π-back-donation. The effects are identified by comparing experimental and simulated spectra related to both the unoccupied and occupied molecular orbitals in solution.
△ Less
Submitted 1 December, 2013;
originally announced December 2013.
-
Stable Estimation of a Covariance Matrix Guided by Nuclear Norm Penalties
Authors:
Eric C. Chi,
Kenneth Lange
Abstract:
Estimation of covariance matrices or their inverses plays a central role in many statistical methods. For these methods to work reliably, estimated matrices must not only be invertible but also well-conditioned. In this paper we present an intuitive prior that shrinks the classic sample covariance estimator towards a stable target. We prove that our estimator is consistent and asymptotically effic…
▽ More
Estimation of covariance matrices or their inverses plays a central role in many statistical methods. For these methods to work reliably, estimated matrices must not only be invertible but also well-conditioned. In this paper we present an intuitive prior that shrinks the classic sample covariance estimator towards a stable target. We prove that our estimator is consistent and asymptotically efficient. Thus, it gracefully transitions towards the sample covariance matrix as the number of samples grows relative to the number of covariates. We also demonstrate the utility of our estimator in two standard situations -- discriminant analysis and EM clustering -- when the number of samples is dominated by or comparable to the number of covariates.
△ Less
Submitted 22 November, 2013; v1 submitted 14 May, 2013;
originally announced May 2013.
-
Splitting Methods for Convex Clustering
Authors:
Eric C. Chi,
Kenneth Lange
Abstract:
Clustering is a fundamental problem in many scientific applications. Standard methods such as $k$-means, Gaussian mixture models, and hierarchical clustering, however, are beset by local minima, which are sometimes drastically suboptimal. Recently introduced convex relaxations of $k$-means and hierarchical clustering shrink cluster centroids toward one another and ensure a unique global minimizer.…
▽ More
Clustering is a fundamental problem in many scientific applications. Standard methods such as $k$-means, Gaussian mixture models, and hierarchical clustering, however, are beset by local minima, which are sometimes drastically suboptimal. Recently introduced convex relaxations of $k$-means and hierarchical clustering shrink cluster centroids toward one another and ensure a unique global minimizer. In this work we present two splitting methods for solving the convex clustering problem. The first is an instance of the alternating direction method of multipliers (ADMM); the second is an instance of the alternating minimization algorithm (AMA). In contrast to previously considered algorithms, our ADMM and AMA formulations provide simple and unified frameworks for solving the convex clustering problem under the previously studied norms and open the door to potentially novel norms. We demonstrate the performance of our algorithm on both simulated and real data examples. While the differences between the two algorithms appear to be minor on the surface, complexity analysis and numerical experiments show AMA to be significantly more efficient.
△ Less
Submitted 18 March, 2014; v1 submitted 1 April, 2013;
originally announced April 2013.
-
A valuation theoretic characterization of recursively saturated real closed fields
Authors:
Paola D'Aquino,
Salma Kuhlmann,
Karen Lange
Abstract:
We give a valuation theoretic characterization for a real closed field to be recursively saturated. Our result extends the characterization of Harnik and Ressayre \cite{hr} for a divisible ordered abelian group to be recursively saturated.
We give a valuation theoretic characterization for a real closed field to be recursively saturated. Our result extends the characterization of Harnik and Ressayre \cite{hr} for a divisible ordered abelian group to be recursively saturated.
△ Less
Submitted 31 December, 2012;
originally announced December 2012.
-
Distance Majorization and Its Applications
Authors:
Eric C. Chi,
Hua Zhou,
Kenneth Lange
Abstract:
The problem of minimizing a continuously differentiable convex function over an intersection of closed convex sets is ubiquitous in applied mathematics. It is particularly interesting when it is easy to project onto each separate set, but nontrivial to project onto their intersection. Algorithms based on Newton's method such as the interior point method are viable for small to medium-scale problem…
▽ More
The problem of minimizing a continuously differentiable convex function over an intersection of closed convex sets is ubiquitous in applied mathematics. It is particularly interesting when it is easy to project onto each separate set, but nontrivial to project onto their intersection. Algorithms based on Newton's method such as the interior point method are viable for small to medium-scale problems. However, modern applications in statistics, engineering, and machine learning are posing problems with potentially tens of thousands of parameters or more. We revisit this convex programming problem and propose an algorithm that scales well with dimensionality. Our proposal is an instance of a sequential unconstrained minimization technique and revolves around three ideas: the majorization-minimization (MM) principle, the classical penalty method for constrained optimization, and quasi-Newton acceleration of fixed-point algorithms. The performance of our distance majorization algorithms is illustrated in several applications.
△ Less
Submitted 11 June, 2013; v1 submitted 16 November, 2012;
originally announced November 2012.
-
Techniques for Solving Sudoku Puzzles
Authors:
Eric C. Chi,
Kenneth Lange
Abstract:
Solving Sudoku puzzles is one of the most popular pastimes in the world. Puzzles range in difficulty from easy to very challenging; the hardest puzzles tend to have the most empty cells. The current paper explains and compares three algorithms for solving Sudoku puzzles. Backtracking, simulated annealing, and alternating projections are generic methods for attacking combinatorial optimization prob…
▽ More
Solving Sudoku puzzles is one of the most popular pastimes in the world. Puzzles range in difficulty from easy to very challenging; the hardest puzzles tend to have the most empty cells. The current paper explains and compares three algorithms for solving Sudoku puzzles. Backtracking, simulated annealing, and alternating projections are generic methods for attacking combinatorial optimization problems. Our results favor backtracking. It infallibly solves a Sudoku puzzle or deduces that a unique solution does not exist. However, backtracking does not scale well in high-dimensional combinatorial optimization. Hence, it is useful to expose students in the mathematical sciences to the other two solution techniques in a concrete setting. Simulated annealing shares a common structure with MCMC (Markov chain Monte Carlo) and enjoys wide applicability. The method of alternating projections solves the feasibility problem in convex programming. Converting a discrete optimization problem into a continuous optimization problem opens up the possibility of handling combinatorial problems of much higher dimensionality.
△ Less
Submitted 16 May, 2013; v1 submitted 10 March, 2012;
originally announced March 2012.
-
A Look at the Generalized Heron Problem through the Lens of Majorization-Minimization
Authors:
Eric C. Chi,
Kenneth Lange
Abstract:
In a recent issue of this journal, Mordukhovich et al.\ pose and solve an interesting non-differentiable generalization of the Heron problem in the framework of modern convex analysis. In the generalized Heron problem one is given $k+1$ closed convex sets in $\Real^d$ equipped with its Euclidean norm and asked to find the point in the last set such that the sum of the distances to the first $k$ se…
▽ More
In a recent issue of this journal, Mordukhovich et al.\ pose and solve an interesting non-differentiable generalization of the Heron problem in the framework of modern convex analysis. In the generalized Heron problem one is given $k+1$ closed convex sets in $\Real^d$ equipped with its Euclidean norm and asked to find the point in the last set such that the sum of the distances to the first $k$ sets is minimal. In later work the authors generalize the Heron problem even further, relax its convexity assumptions, study its theoretical properties, and pursue subgradient algorithms for solving the convex case. Here, we revisit the original problem solely from the numerical perspective. By exploiting the majorization-minimization (MM) principle of computational statistics and rudimentary techniques from differential calculus, we are able to construct a very fast algorithm for solving the Euclidean version of the generalized Heron problem.
△ Less
Submitted 23 May, 2012; v1 submitted 2 March, 2012;
originally announced March 2012.
-
Reconstructing DNA copy number by joint segmentation of multiple sequences
Authors:
Zhongyang Zhang,
Kenneth Lange,
Chiara Sabatti
Abstract:
The variation in DNA copy number carries information on the modalities of genome evolution and misregulation of DNA replication in cancer cells; its study can be helpful to localize tumor suppressor genes, distinguish different populations of cancerous cell, as well identify genomic variations responsible for disease phenotypes. A number of different high throughput technologies can be used to ide…
▽ More
The variation in DNA copy number carries information on the modalities of genome evolution and misregulation of DNA replication in cancer cells; its study can be helpful to localize tumor suppressor genes, distinguish different populations of cancerous cell, as well identify genomic variations responsible for disease phenotypes. A number of different high throughput technologies can be used to identify copy number variable sites, and the literature documents multiple effective algorithms. We focus here on the specific problem of detecting regions where variation in copy number is relatively common in the sample at hand: this encompasses the cases of copy number polymorphisms, related samples, technical replicates, and cancerous sub-populations from the same individual. We present an algorithm based on regularization approaches with significant computational advantages and competitive accuracy. We illustrate its applicability with simulated and real data sets.
△ Less
Submitted 14 March, 2012; v1 submitted 22 February, 2012;
originally announced February 2012.
-
Path Following in the Exact Penalty Method of Convex Programming
Authors:
Hua Zhou,
Kenneth Lange
Abstract:
Classical penalty methods solve a sequence of unconstrained problems that put greater and greater stress on meeting the constraints. In the limit as the penalty constant tends to $\infty$, one recovers the constrained solution. In the exact penalty method, squared penalties are replaced by absolute value penalties, and the solution is recovered for a finite value of the penalty constant. In practi…
▽ More
Classical penalty methods solve a sequence of unconstrained problems that put greater and greater stress on meeting the constraints. In the limit as the penalty constant tends to $\infty$, one recovers the constrained solution. In the exact penalty method, squared penalties are replaced by absolute value penalties, and the solution is recovered for a finite value of the penalty constant. In practice, the kinks in the penalty and the unknown magnitude of the penalty constant prevent wide application of the exact penalty method in nonlinear programming. In this article, we examine a strategy of path following consistent with the exact penalty method. Instead of performing optimization at a single penalty constant, we trace the solution as a continuous function of the penalty constant. Thus, path following starts at the unconstrained solution and follows the solution path as the penalty constant increases. In the process, the solution path hits, slides along, and exits from the various constraints. For quadratic programming, the solution path is piecewise linear and takes large jumps from constraint to constraint. For a general convex program, the solution path is piecewise smooth, and path following operates by numerically solving an ordinary differential equation segment by segment. Our diverse applications to a) projection onto a convex set, b) nonnegative least squares, c) quadratically constrained quadratic programming, d) geometric programming, and e) semidefinite programming illustrate the mechanics and potential of path following. The final detour to image denoising demonstrates the relevance of path following to regularized estimation in inverse problems. In regularized estimation, one follows the solution path as the penalty constant decreases from a large value.
△ Less
Submitted 17 January, 2012;
originally announced January 2012.
-
Real closed exponential fields
Authors:
Paola D'Aquino,
Julia F. Knight,
Salma Kuhlmann,
Karen Lange
Abstract:
In an extended abstract Ressayre considered real closed exponential fields and integer parts that respect the exponential function. He outlined a proof that every real closed exponential field has an exponential integer part. In the present paper, we give a detailed account of Ressayre's construction, which becomes canonical once we fix the real closed exponential field, a residue field section, a…
▽ More
In an extended abstract Ressayre considered real closed exponential fields and integer parts that respect the exponential function. He outlined a proof that every real closed exponential field has an exponential integer part. In the present paper, we give a detailed account of Ressayre's construction, which becomes canonical once we fix the real closed exponential field, a residue field section, and a well ordering of the field. The procedure is constructible over these objects; each step looks effective, but may require many steps. We produce an example of an exponential field $R$ with a residue field $k$ and a well ordering $<$ such that $D^c(R)$ is low and $k$ and $<$ are $Δ^0_3$, and Ressayre's construction cannot be completed in $L_{ω_1^{CK}}$.
△ Less
Submitted 17 December, 2011;
originally announced December 2011.
-
The MM Alternative to EM
Authors:
Tong Tong Wu,
Kenneth Lange
Abstract:
The EM algorithm is a special case of a more general algorithm called the MM algorithm. Specific MM algorithms often have nothing to do with missing data. The first M step of an MM algorithm creates a surrogate function that is optimized in the second M step. In minimization, MM stands for majorize--minimize; in maximization, it stands for minorize--maximize. This two-step process always drives th…
▽ More
The EM algorithm is a special case of a more general algorithm called the MM algorithm. Specific MM algorithms often have nothing to do with missing data. The first M step of an MM algorithm creates a surrogate function that is optimized in the second M step. In minimization, MM stands for majorize--minimize; in maximization, it stands for minorize--maximize. This two-step process always drives the objective function in the right direction. Construction of MM algorithms relies on recognizing and manipulating inequalities rather than calculating conditional expectations. This survey walks the reader through the construction of several specific MM algorithms. The potential of the MM algorithm in solving high-dimensional optimization and estimation problems is its most attractive feature. Our applications to random graph models, discriminant analysis and image restoration showcase this ability.
△ Less
Submitted 12 April, 2011;
originally announced April 2011.
-
A Path Algorithm for Constrained Estimation
Authors:
Hua Zhou,
Kenneth Lange
Abstract:
Many least squares problems involve affine equality and inequality constraints. Although there are variety of methods for solving such problems, most statisticians find constrained estimation challenging. The current paper proposes a new path following algorithm for quadratic programming based on exact penalization. Similar penalties arise in $l_1$ regularization in model selection. Classical pena…
▽ More
Many least squares problems involve affine equality and inequality constraints. Although there are variety of methods for solving such problems, most statisticians find constrained estimation challenging. The current paper proposes a new path following algorithm for quadratic programming based on exact penalization. Similar penalties arise in $l_1$ regularization in model selection. Classical penalty methods solve a sequence of unconstrained problems that put greater and greater stress on meeting the constraints. In the limit as the penalty constant tends to $\infty$, one recovers the constrained solution. In the exact penalty method, squared penalties are replaced by absolute value penalties, and the solution is recovered for a finite value of the penalty constant. The exact path following method starts at the unconstrained solution and follows the solution path as the penalty constant increases. In the process, the solution path hits, slides along, and exits from the various constraints. Path following in lasso penalized regression, in contrast, starts with a large value of the penalty constant and works its way downward. In both settings, inspection of the entire solution path is revealing. Just as with the lasso and generalized lasso, it is possible to plot the effective degrees of freedom along the solution path. For a strictly convex quadratic program, the exact penalty algorithm can be framed entirely in terms of the sweep operator of regression analysis. A few well chosen examples illustrate the mechanics and potential of path following.
△ Less
Submitted 18 March, 2011;
originally announced March 2011.
-
Multicategory vertex discriminant analysis for high-dimensional data
Authors:
Tong Tong Wu,
Kenneth Lange
Abstract:
In response to the challenges of data mining, discriminant analysis continues to evolve as a vital branch of statistics. Our recently introduced method of vertex discriminant analysis (VDA) is ideally suited to handle multiple categories and an excess of predictors over training cases. The current paper explores an elaboration of VDA that conducts classification and variable selection simultaneous…
▽ More
In response to the challenges of data mining, discriminant analysis continues to evolve as a vital branch of statistics. Our recently introduced method of vertex discriminant analysis (VDA) is ideally suited to handle multiple categories and an excess of predictors over training cases. The current paper explores an elaboration of VDA that conducts classification and variable selection simultaneously. Adding lasso ($\ell_1$-norm) and Euclidean penalties to the VDA loss function eliminates unnecessary predictors. Lasso penalties apply to each predictor coefficient separately; Euclidean penalties group the collective coefficients of a single predictor. With these penalties in place, cyclic coordinate descent accelerates estimation of all coefficients. Our tests on simulated and benchmark real data demonstrate the virtues of penalized VDA in model building and prediction in high-dimensional settings.
△ Less
Submitted 5 January, 2011;
originally announced January 2011.
-
On n-Tardy Sets
Authors:
Peter A. Cholak,
Peter M. Gerdes,
Karen Lange
Abstract:
Harrington and Soare introduced the notion of an n-tardy set. They showed that there is a nonempty $\mathcal{E}$ property Q(A) such that if Q(A) then A is 2-tardy. Since they also showed no 2-tardy set is complete, Harrington and Soare showed that there exists an orbit of computably enumerable sets such that every set in that orbit is incomplete. Our study of n-tardy sets takes off from where Harr…
▽ More
Harrington and Soare introduced the notion of an n-tardy set. They showed that there is a nonempty $\mathcal{E}$ property Q(A) such that if Q(A) then A is 2-tardy. Since they also showed no 2-tardy set is complete, Harrington and Soare showed that there exists an orbit of computably enumerable sets such that every set in that orbit is incomplete. Our study of n-tardy sets takes off from where Harrington and Soare left off. We answer all the open questions asked by Harrington and Soare about n-tardy sets. We show there is a 3-tardy set A that is not computed by any 2-tardy set B. We also show that there are nonempty $\mathcal{E}$ properties $Q_n(A)$ such that if $Q_n(A)$ then A is properly n-tardy.
△ Less
Submitted 31 December, 2010;
originally announced January 2011.
-
MM Algorithms for Geometric and Signomial Programming
Authors:
Kenneth Lange,
Hua Zhou
Abstract:
This paper derives new algorithms for signomial programming, a generalization of geometric programming. The algorithms are based on a generic principle for optimization called the MM algorithm. In this setting, one can apply the geometric-arithmetic mean inequality and a supporting hyperplane inequality to create a surrogate function with parameters separated. Thus, unconstrained signomial program…
▽ More
This paper derives new algorithms for signomial programming, a generalization of geometric programming. The algorithms are based on a generic principle for optimization called the MM algorithm. In this setting, one can apply the geometric-arithmetic mean inequality and a supporting hyperplane inequality to create a surrogate function with parameters separated. Thus, unconstrained signomial programming reduces to a sequence of one-dimensional minimization problems. Simple examples demonstrate that the MM algorithm derived can converge to a boundary point or to one point of a continuum of minimum points. Conditions under which the minimum point is unique or occurs in the interior of parameter space are proved for geometric programming. Convergence to an interior point occurs at a linear rate. Finally, the MM framework easily accommodates equality and inequality constraints of signomial type. For the most important special case, constrained quadratic programming, the MM algorithm involves very simple updates.
△ Less
Submitted 14 July, 2010;
originally announced July 2010.
-
Graphics Processing Units and High-Dimensional Optimization
Authors:
Hua Zhou,
Kenneth Lange,
Marc A. Suchard
Abstract:
This paper discusses the potential of graphics processing units (GPUs) in high-dimensional optimization problems. A single GPU card with hundreds of arithmetic cores can be inserted in a personal computer and dramatically accelerates many statistical algorithms. To exploit these devices fully, optimization algorithms should reduce to multiple parallel tasks, each accessing a limited amount of dat…
▽ More
This paper discusses the potential of graphics processing units (GPUs) in high-dimensional optimization problems. A single GPU card with hundreds of arithmetic cores can be inserted in a personal computer and dramatically accelerates many statistical algorithms. To exploit these devices fully, optimization algorithms should reduce to multiple parallel tasks, each accessing a limited amount of data. These criteria favor EM and MM algorithms that separate parameters and data. To a lesser extent block relaxation and coordinate descent and ascent also qualify. We demonstrate the utility of GPUs in nonnegative matrix factorization, PET image reconstruction, and multidimensional scaling. Speedups of 100 fold can easily be attained. Over the next decade, GPUs will fundamentally alter the landscape of computational statistics. It is time for more statisticians to get on-board.
△ Less
Submitted 16 March, 2010;
originally announced March 2010.
-
A virtual substitution of Brouwer choice sequence
Authors:
Klaus Lange
Abstract:
Step by step a substitution of the well known Brouwer choice sequence will be constructed. It begins with an establishing of quasi alternating prime number series followed by a construction of a virtual sequence in sense of the virtual set definition. The last step gives reasons for why this virtual sequence substitutes the choice sequence created by L. E. J. Brouwer.
Step by step a substitution of the well known Brouwer choice sequence will be constructed. It begins with an establishing of quasi alternating prime number series followed by a construction of a virtual sequence in sense of the virtual set definition. The last step gives reasons for why this virtual sequence substitutes the choice sequence created by L. E. J. Brouwer.
△ Less
Submitted 24 August, 2009;
originally announced August 2009.
-
Reconstructing DNA copy number by penalized estimation and imputation
Authors:
Zhongyang Zhang,
Kenneth Lange,
Roel Ophoff,
Chiara Sabatti
Abstract:
Recent advances in genomics have underscored the surprising ubiquity of DNA copy number variation (CNV). Fortunately, modern genotyping platforms also detect CNVs with fairly high reliability. Hidden Markov models and algorithms have played a dominant role in the interpretation of CNV data. Here we explore CNV reconstruction via estimation with a fused-lasso penalty as suggested by Tibshirani and…
▽ More
Recent advances in genomics have underscored the surprising ubiquity of DNA copy number variation (CNV). Fortunately, modern genotyping platforms also detect CNVs with fairly high reliability. Hidden Markov models and algorithms have played a dominant role in the interpretation of CNV data. Here we explore CNV reconstruction via estimation with a fused-lasso penalty as suggested by Tibshirani and Wang [Biostatistics 9 (2008) 18--29]. We mount a fresh attack on this difficult optimization problem by the following: (a) changing the penalty terms slightly by substituting a smooth approximation to the absolute value function, (b) designing and implementing a new MM (majorization--minimization) algorithm, and (c) applying a fast version of Newton's method to jointly update all model parameters. Together these changes enable us to minimize the fused-lasso criterion in a highly effective way. We also reframe the reconstruction problem in terms of imputation via discrete optimization. This approach is easier and more accurate than parameter estimation because it relies on the fact that only a handful of possible copy number states exist at each SNP. The dynamic programming framework has the added bonus of exploiting information that the current fused-lasso approach ignores. The accuracy of our imputations is comparable to that of hidden Markov models at a substantially lower computational cost.
△ Less
Submitted 10 January, 2011; v1 submitted 11 June, 2009;
originally announced June 2009.
-
About a virtual subset
Authors:
Klaus Lange
Abstract:
Two constructed prime number subsets (called prime brother & sisters and prime cousins) lead to a third one (called isolated primes) so that all three disjoint subsets together generate the prime number set. It should be suggested how the subset of isolated primes give a new approach to expand the set theory by using virtual subsets.
Two constructed prime number subsets (called prime brother & sisters and prime cousins) lead to a third one (called isolated primes) so that all three disjoint subsets together generate the prime number set. It should be suggested how the subset of isolated primes give a new approach to expand the set theory by using virtual subsets.
△ Less
Submitted 30 April, 2009;
originally announced April 2009.
-
Coordinate descent algorithms for lasso penalized regression
Authors:
Tong Tong Wu,
Kenneth Lange
Abstract:
Imposition of a lasso penalty shrinks parameter estimates toward zero and performs continuous model selection. Lasso penalized regression is capable of handling linear regression problems where the number of predictors far exceeds the number of cases. This paper tests two exceptionally fast algorithms for estimating regression coefficients with a lasso penalty. The previously known $\ell_2$ algo…
▽ More
Imposition of a lasso penalty shrinks parameter estimates toward zero and performs continuous model selection. Lasso penalized regression is capable of handling linear regression problems where the number of predictors far exceeds the number of cases. This paper tests two exceptionally fast algorithms for estimating regression coefficients with a lasso penalty. The previously known $\ell_2$ algorithm is based on cyclic coordinate descent. Our new $\ell_1$ algorithm is based on greedy coordinate descent and Edgeworth's algorithm for ordinary $\ell_1$ regression. Each algorithm relies on a tuning constant that can be chosen by cross-validation. In some regression problems it is natural to group parameters and penalize parameters group by group rather than separately. If the group penalty is proportional to the Euclidean norm of the parameters of the group, then it is possible to majorize the norm and reduce parameter estimation to $\ell_2$ regression with a lasso penalty. Thus, the existing algorithm can be extended to novel settings. Each of the algorithms discussed is tested via either simulated or real data or both. The Appendix proves that a greedy form of the $\ell_2$ algorithm converges to the minimum value of the objective function.
△ Less
Submitted 27 March, 2008;
originally announced March 2008.
