-
An iterative algorithm for the square-root Lasso
Authors:
Patrizia Boccacci,
Christine De Mol,
Ignace Loris
Abstract:
In the framework of sparsity-enforcing regularisation for linear inverse problems, we consider the minimisation of a square-root Lasso cost function. We show that a simple modification of the Iterative Soft-Thresholding Algorithm (ISTA) for the Lasso problem leads to a minimiser of this cost function and we investigate the convergence properties of this algorithm. We also consider the group square…
▽ More
In the framework of sparsity-enforcing regularisation for linear inverse problems, we consider the minimisation of a square-root Lasso cost function. We show that a simple modification of the Iterative Soft-Thresholding Algorithm (ISTA) for the Lasso problem leads to a minimiser of this cost function and we investigate the convergence properties of this algorithm. We also consider the group square-root Lasso, where sparsity is enforced for groups of variables instead of individual ones.
△ Less
Submitted 28 March, 2025;
originally announced March 2025.
-
Proceedings of the second "international Traveling Workshop on Interactions between Sparse models and Technology" (iTWIST'14)
Authors:
L. Jacques,
C. De Vleeschouwer,
Y. Boursier,
P. Sudhakar,
C. De Mol,
A. Pizurica,
S. Anthoine,
P. Vandergheynst,
P. Frossard,
C. Bilen,
S. Kitic,
N. Bertin,
R. Gribonval,
N. Boumal,
B. Mishra,
P. -A. Absil,
R. Sepulchre,
S. Bundervoet,
C. Schretter,
A. Dooms,
P. Schelkens,
O. Chabiron,
F. Malgouyres,
J. -Y. Tourneret,
N. Dobigeon
, et al. (42 additional authors not shown)
Abstract:
The implicit objective of the biennial "international - Traveling Workshop on Interactions between Sparse models and Technology" (iTWIST) is to foster collaboration between international scientific teams by disseminating ideas through both specific oral/poster presentations and free discussions. For its second edition, the iTWIST workshop took place in the medieval and picturesque town of Namur in…
▽ More
The implicit objective of the biennial "international - Traveling Workshop on Interactions between Sparse models and Technology" (iTWIST) is to foster collaboration between international scientific teams by disseminating ideas through both specific oral/poster presentations and free discussions. For its second edition, the iTWIST workshop took place in the medieval and picturesque town of Namur in Belgium, from Wednesday August 27th till Friday August 29th, 2014. The workshop was conveniently located in "The Arsenal" building within walking distance of both hotels and town center. iTWIST'14 has gathered about 70 international participants and has featured 9 invited talks, 10 oral presentations, and 14 posters on the following themes, all related to the theory, application and generalization of the "sparsity paradigm": Sparsity-driven data sensing and processing; Union of low dimensional subspaces; Beyond linear and convex inverse problem; Matrix/manifold/graph sensing/processing; Blind inverse problems and dictionary learning; Sparsity and computational neuroscience; Information theory, geometry and randomness; Complexity/accuracy tradeoffs in numerical methods; Sparsity? What's next?; Sparse machine learning and inference.
△ Less
Submitted 9 October, 2014; v1 submitted 2 October, 2014;
originally announced October 2014.
-
Accelerating gradient projection methods for $\ell_1$-constrained signal recovery by steplength selection rules
Authors:
I. Loris,
M. Bertero,
C. De Mol,
R. Zanella,
L. Zanni
Abstract:
We propose a new gradient projection algorithm that compares favorably with the fastest algorithms available to date for $\ell_1$-constrained sparse recovery from noisy data, both in the compressed sensing and inverse problem frameworks. The method exploits a line-search along the feasible direction and an adaptive steplength selection based on recent strategies for the alternation of the well-k…
▽ More
We propose a new gradient projection algorithm that compares favorably with the fastest algorithms available to date for $\ell_1$-constrained sparse recovery from noisy data, both in the compressed sensing and inverse problem frameworks. The method exploits a line-search along the feasible direction and an adaptive steplength selection based on recent strategies for the alternation of the well-known Barzilai-Borwein rules. The convergence of the proposed approach is discussed and a computational study on both well-conditioned and ill-conditioned problems is carried out for performance evaluations in comparison with five other algorithms proposed in the literature.
△ Less
Submitted 25 February, 2009;
originally announced February 2009.
-
Elastic-Net Regularization in Learning Theory
Authors:
C. De Mol,
E. De Vito,
L. Rosasco
Abstract:
Within the framework of statistical learning theory we analyze in detail the so-called elastic-net regularization scheme proposed by Zou and Hastie for the selection of groups of correlated variables. To investigate on the statistical properties of this scheme and in particular on its consistency properties, we set up a suitable mathematical framework. Our setting is random-design regression whe…
▽ More
Within the framework of statistical learning theory we analyze in detail the so-called elastic-net regularization scheme proposed by Zou and Hastie for the selection of groups of correlated variables. To investigate on the statistical properties of this scheme and in particular on its consistency properties, we set up a suitable mathematical framework. Our setting is random-design regression where we allow the response variable to be vector-valued and we consider prediction functions which are linear combination of elements ({\em features}) in an infinite-dimensional dictionary. Under the assumption that the regression function admits a sparse representation on the dictionary, we prove that there exists a particular ``{\em elastic-net representation}'' of the regression function such that, if the number of data increases, the elastic-net estimator is consistent not only for prediction but also for variable/feature selection. Our results include finite-sample bounds and an adaptive scheme to select the regularization parameter. Moreover, using convex analysis tools, we derive an iterative thresholding algorithm for computing the elastic-net solution which is different from the optimization procedure originally proposed by Zou and Hastie
△ Less
Submitted 22 July, 2008;
originally announced July 2008.
-
Sparse and stable Markowitz portfolios
Authors:
Joshua Brodie,
Ingrid Daubechies,
Christine De Mol,
Domenico Giannone,
Ignace Loris
Abstract:
We consider the problem of portfolio selection within the classical Markowitz mean-variance framework, reformulated as a constrained least-squares regression problem. We propose to add to the objective function a penalty proportional to the sum of the absolute values of the portfolio weights. This penalty regularizes (stabilizes) the optimization problem, encourages sparse portfolios (i.e. portf…
▽ More
We consider the problem of portfolio selection within the classical Markowitz mean-variance framework, reformulated as a constrained least-squares regression problem. We propose to add to the objective function a penalty proportional to the sum of the absolute values of the portfolio weights. This penalty regularizes (stabilizes) the optimization problem, encourages sparse portfolios (i.e. portfolios with only few active positions), and allows to account for transaction costs. Our approach recovers as special cases the no-short-positions portfolios, but does allow for short positions in limited number. We implement this methodology on two benchmark data sets constructed by Fama and French. Using only a modest amount of training data, we construct portfolios whose out-of-sample performance, as measured by Sharpe ratio, is consistently and significantly better than that of the naive evenly-weighted portfolio which constitutes, as shown in recent literature, a very tough benchmark.
△ Less
Submitted 29 May, 2008; v1 submitted 31 July, 2007;
originally announced August 2007.
-
An iterative thresholding algorithm for linear inverse problems with a sparsity constraint
Authors:
Ingrid Daubechies,
Michel Defrise,
Christine De Mol
Abstract:
We consider linear inverse problems where the solution is assumed to have a sparse expansion on an arbitrary pre-assigned orthonormal basis. We prove that replacing the usual quadratic regularizing penalties by weighted l^p-penalties on the coefficients of such expansions, with 1 < or = p < or =2, still regularizes the problem. If p < 2, regularized solutions of such l^p-penalized problems will…
▽ More
We consider linear inverse problems where the solution is assumed to have a sparse expansion on an arbitrary pre-assigned orthonormal basis. We prove that replacing the usual quadratic regularizing penalties by weighted l^p-penalties on the coefficients of such expansions, with 1 < or = p < or =2, still regularizes the problem. If p < 2, regularized solutions of such l^p-penalized problems will have sparser expansions, with respect to the basis under consideration. To compute the corresponding regularized solutions we propose an iterative algorithm that amounts to a Landweber iteration with thresholding (or nonlinear shrinkage) applied at each iteration step. We prove that this algorithm converges in norm. We also review some potential applications of this method.
△ Less
Submitted 2 November, 2003; v1 submitted 10 July, 2003;
originally announced July 2003.
