-
Variable Bregman Majorization-Minimization Algorithm and its Application to Dirichlet Maximum Likelihood Estimation
Authors:
Ségolène Martin,
Jean-Christophe Pesquet,
Gabriele Steidl,
Ismail Ben Ayed
Abstract:
We propose a novel Bregman descent algorithm for minimizing a convex function that is expressed as the sum of a differentiable part (defined over an open set) and a possibly nonsmooth term. The approach, referred to as the Variable Bregman Majorization-Minimization (VBMM) algorithm, extends the Bregman Proximal Gradient method by allowing the Bregman function used in the divergence to adaptively v…
▽ More
We propose a novel Bregman descent algorithm for minimizing a convex function that is expressed as the sum of a differentiable part (defined over an open set) and a possibly nonsmooth term. The approach, referred to as the Variable Bregman Majorization-Minimization (VBMM) algorithm, extends the Bregman Proximal Gradient method by allowing the Bregman function used in the divergence to adaptively vary at each iteration, provided it satisfies a majorizing condition on the objective function. This adaptive framework enables the algorithm to approximate the objective more precisely at each iteration, thereby allowing for accelerated convergence compared to the traditional Bregman Proximal Gradient descent. We establish the convergence of the VBMM algorithm to a minimizer under mild assumptions on the family of metrics used. Furthermore, we introduce a novel application of both the Bregman Proximal Gradient method and the VBMM algorithm to the estimation of the multidimensional parameters of a Dirichlet distribution through the maximization of its log-likelihood. Numerical experiments confirm that the VBMM algorithm outperforms existing approaches in terms of convergence speed.
△ Less
Submitted 5 February, 2025; v1 submitted 13 January, 2025;
originally announced January 2025.
-
Towards Practical Few-Shot Query Sets: Transductive Minimum Description Length Inference
Authors:
Ségolène Martin,
Malik Boudiaf,
Emilie Chouzenoux,
Jean-Christophe Pesquet,
Ismail Ben Ayed
Abstract:
Standard few-shot benchmarks are often built upon simplifying assumptions on the query sets, which may not always hold in practice. In particular, for each task at testing time, the classes effectively present in the unlabeled query set are known a priori, and correspond exactly to the set of classes represented in the labeled support set. We relax these assumptions and extend current benchmarks,…
▽ More
Standard few-shot benchmarks are often built upon simplifying assumptions on the query sets, which may not always hold in practice. In particular, for each task at testing time, the classes effectively present in the unlabeled query set are known a priori, and correspond exactly to the set of classes represented in the labeled support set. We relax these assumptions and extend current benchmarks, so that the query-set classes of a given task are unknown, but just belong to a much larger set of possible classes. Our setting could be viewed as an instance of the challenging yet practical problem of extremely imbalanced K-way classification, K being much larger than the values typically used in standard benchmarks, and with potentially irrelevant supervision from the support set. Expectedly, our setting incurs drops in the performances of state-of-the-art methods. Motivated by these observations, we introduce a PrimAl Dual Minimum Description LEngth (PADDLE) formulation, which balances data-fitting accuracy and model complexity for a given few-shot task, under supervision constraints from the support set. Our constrained MDL-like objective promotes competition among a large set of possible classes, preserving only effective classes that befit better the data of a few-shot task. It is hyperparameter free, and could be applied on top of any base-class training. Furthermore, we derive a fast block coordinate descent algorithm for optimizing our objective, with convergence guarantee, and a linear computational complexity at each iteration. Comprehensive experiments over the standard few-shot datasets and the more realistic and challenging i-Nat dataset show highly competitive performances of our method, more so when the numbers of possible classes in the tasks increase. Our code is publicly available at https://github.com/SegoleneMartin/PADDLE.
△ Less
Submitted 26 October, 2022;
originally announced October 2022.
-
A Preconditioned Alternating Minimization Framework for Nonconvex and Half Quadratic Optimization
Authors:
Shengxiang Deng,
Ismail Ben Ayed,
Hongpeng Sun
Abstract:
For some typical and widely used non-convex half-quadratic regularization models and the Ambrosio-Tortorelli approximate Mumford-Shah model, based on the Kurdyka-Łojasiewicz analysis and the recent nonconvex proximal algorithms, we developed an efficient preconditioned framework aiming at the linear subproblems that appeared in the nonlinear alternating minimization procedure. Solving large-scale…
▽ More
For some typical and widely used non-convex half-quadratic regularization models and the Ambrosio-Tortorelli approximate Mumford-Shah model, based on the Kurdyka-Łojasiewicz analysis and the recent nonconvex proximal algorithms, we developed an efficient preconditioned framework aiming at the linear subproblems that appeared in the nonlinear alternating minimization procedure. Solving large-scale linear subproblems is always important and challenging for lots of alternating minimization algorithms. By cooperating the efficient and classical preconditioned iterations into the nonlinear and nonconvex optimization, we prove that only one or any finite times preconditioned iterations are needed for the linear subproblems without controlling the error as the usual inexact solvers. The proposed preconditioned framework can provide great flexibility and efficiency for dealing with linear subproblems and guarantee the global convergence of the nonlinear alternating minimization method simultaneously.
△ Less
Submitted 29 July, 2021;
originally announced July 2021.
-
Description of the lack of compactness in Orlicz spaces and applications
Authors:
Ines Ben Ayed,
Mohamed Khalil Zghal
Abstract:
In this paper, we investigate the lack of compactness of the Sobolev embedding of $H^1(\R^2)$ into the Orlicz space $L^{φ_p}(\R^2)$ associated to the function $φ_p$ defined by $φ_p(s):={\rm{e}^{s^2}}-\Sum_{k=0}^{p-1} \frac{s^{2k}}{k!}\cdot$ We also undertake the study of a nonlinear wave equation with exponential growth where the Orlicz norm $\|.\|_{L^{φ_p}}$ plays a crucial role. This study inclu…
▽ More
In this paper, we investigate the lack of compactness of the Sobolev embedding of $H^1(\R^2)$ into the Orlicz space $L^{φ_p}(\R^2)$ associated to the function $φ_p$ defined by $φ_p(s):={\rm{e}^{s^2}}-\Sum_{k=0}^{p-1} \frac{s^{2k}}{k!}\cdot$ We also undertake the study of a nonlinear wave equation with exponential growth where the Orlicz norm $\|.\|_{L^{φ_p}}$ plays a crucial role. This study includes issues of global existence, scattering and qualitative study.
△ Less
Submitted 21 December, 2013;
originally announced December 2013.
-
Characterization of the lack of compactness of $H^2_{rad}(\R^4)$ into the Orlicz space
Authors:
Ines Ben Ayed,
Mohamed Khalil Zghal
Abstract:
This paper is devoted to the description of the lack of compactness of the Sobolev space $H^2_{rad}(\R^4)$ in the Orlicz space $\mathcal{L}(\R^4)$. The approach that we adopt to establish this characterization is in the spirit of the one adopted in the case of $H^1_{rad}(\R^2)$ into the Orlicz space $\mathcal{L}(\R^2)$ in \cite{Bahouri}.
This paper is devoted to the description of the lack of compactness of the Sobolev space $H^2_{rad}(\R^4)$ in the Orlicz space $\mathcal{L}(\R^4)$. The approach that we adopt to establish this characterization is in the spirit of the one adopted in the case of $H^1_{rad}(\R^2)$ into the Orlicz space $\mathcal{L}(\R^2)$ in \cite{Bahouri}.
△ Less
Submitted 18 January, 2013;
originally announced January 2013.
