-
How Stringent is the Linear Independence Kink Qualification in Abs-Smooth Optimization?
Authors:
Lukas Baumgärtner,
Franz Bethke,
Ganna Shyshkanova,
Andrea Walther
Abstract:
Abs-smooth functions are given by compositions of smooth functions and the evaluation of the absolute value. The linear independence kink qualification (LIKQ) is a fundamental assumption in optimization problems governed by these abs-smooth functions, generalizing the well-known LICQ from smooth optimization. In particular, provided that LIKQ holds it is possible to derive optimality conditions fo…
▽ More
Abs-smooth functions are given by compositions of smooth functions and the evaluation of the absolute value. The linear independence kink qualification (LIKQ) is a fundamental assumption in optimization problems governed by these abs-smooth functions, generalizing the well-known LICQ from smooth optimization. In particular, provided that LIKQ holds it is possible to derive optimality conditions for abs-smooth optimization problems that can be checked in polynomial time. Utilizing tools from differential topology, namely a version of the jet-transversality theorem, it is shown that assuming LIKQ for all feasible points of an abs-smooth optimization problem is a generic assumption.
△ Less
Submitted 12 December, 2024;
originally announced December 2024.
-
Two Models for Surface Segmentation using the Total Variation of the Normal Vector
Authors:
Lukas Baumgärtner,
Ronny Bergmann,
Roland Herzog,
Stephan Schmidt,
Manuel Weiß
Abstract:
We consider the problem of surface segmentation, where the goal is to partition a surface represented by a triangular mesh. The segmentation is based on the similarity of the normal vector field to a given set of label vectors. We propose a variational approach and compare two different regularizers, both based on a total variation measure. The first regularizer penalizes the total variation of th…
▽ More
We consider the problem of surface segmentation, where the goal is to partition a surface represented by a triangular mesh. The segmentation is based on the similarity of the normal vector field to a given set of label vectors. We propose a variational approach and compare two different regularizers, both based on a total variation measure. The first regularizer penalizes the total variation of the assignment function directly, while the second regularizer penalizes the total variation in the label space. In order to solve the resulting optimization problems, we use variations of the split Bregman (ADMM) iteration adapted to the problem at hand. While computationally more expensive, the second regularizer yields better results in our experiments, in particular it removes noise more reliably in regions of constant curvature.
△ Less
Submitted 30 November, 2024;
originally announced December 2024.
-
Computation of Generalized Derivatives for Abs-Smooth Functions by Backward Mode Algorithmic Differentiation and Implications to Deep Learning
Authors:
Lukas Baumgärtner,
Franz Bethke
Abstract:
Algorithmic differentiation (AD) tools allow to obtain gradient information of a continuously differentiable objective function in a computationally cheap way using the so-called backward mode. It is common practice to use the same tools even in the absence of differentiability, although the resulting vectors may not be generalized gradients in the sense of Clarke. The paper at hand focuses on obj…
▽ More
Algorithmic differentiation (AD) tools allow to obtain gradient information of a continuously differentiable objective function in a computationally cheap way using the so-called backward mode. It is common practice to use the same tools even in the absence of differentiability, although the resulting vectors may not be generalized gradients in the sense of Clarke. The paper at hand focuses on objectives in which the non-differentiability arises solely from the evaluation of the absolute value function. In that case, an algebraic condition based on the evaluation procedure of the objective is identified, that guarantees that Clarke gradients are correctly computed without requiring any modifications of the AD tool in question. The analysis allows to prove that any standard AD tool is adequate to drive a stochastic generalized gradient descent method for training a dense neural network with ReLU activations. The same is true for generalized batch gradients or the full generalized gradient, provided that the AD tool makes a deterministic and agnostic choice for the derivative information of the absolute value at 0.
△ Less
Submitted 28 November, 2024; v1 submitted 12 July, 2024;
originally announced July 2024.
-
A Multi-Parameter Singular Perturbation Analysis of the Robertson Model
Authors:
Lukas Baumgartner,
Peter Szmolyan
Abstract:
The Robertson model describing a chemical reaction involving three reactants is one of the classical examples of stiffness in ODEs. The stiffness is caused by the occurrence of three reaction rates $k_1,\,k_2$, and $k_3$, with largely differing orders of magnitude, acting as parameters. The model has been widely used as a numerical test problem. Surprisingly, no asymptotic analysis of this multisc…
▽ More
The Robertson model describing a chemical reaction involving three reactants is one of the classical examples of stiffness in ODEs. The stiffness is caused by the occurrence of three reaction rates $k_1,\,k_2$, and $k_3$, with largely differing orders of magnitude, acting as parameters. The model has been widely used as a numerical test problem. Surprisingly, no asymptotic analysis of this multiscale problem seems to exist. In this paper we provide a full asymptotic analysis of the Robertson model under the assumption $k_1, k_3 \ll k_2$. We rewrite the equations as a two-parameter singular perturbation problem in the rescaled small parameters $(\varepsilon_1,\varepsilon_2):=(k_1/k_2,k_3/k_2)$, which we then analyze using geometric singular perturbation theory (GSPT). To deal with the multi-parameter singular structure, we perform blow-ups in parameter- and variable space. We identify four distinct regimes in a neighbourhood of the singular limit $(\varepsilon_1,\varepsilon_2)= (0,0)$. Within these four regimes we use GSPT and additional blow-ups to analyze the dynamics and the structure of solutions. Our asymptotic results are in excellent qualitative and quantitative agreement with the numerics.
△ Less
Submitted 6 February, 2025; v1 submitted 4 July, 2024;
originally announced July 2024.
-
Medical Image Registration using optimal control of a linear hyperbolic transport equation with a DG discretization
Authors:
Bastian Zapf,
Johannes Haubner,
Lukas Baumgärtner,
Stephan Schmidt
Abstract:
Patient specific brain mesh generation from MRI can be a time consuming task and require manual corrections, e.g., for meshing the ventricular system or defining subdomains. To address this issue, we consider an image registration approach. The idea is to use the registration of an input magnetic resonance image (MRI) to a respective target in order to obtain a new mesh from a template mesh. To ob…
▽ More
Patient specific brain mesh generation from MRI can be a time consuming task and require manual corrections, e.g., for meshing the ventricular system or defining subdomains. To address this issue, we consider an image registration approach. The idea is to use the registration of an input magnetic resonance image (MRI) to a respective target in order to obtain a new mesh from a template mesh. To obtain the transformation, we solve an optimization problem that is constrained by a linear hyperbolic transport equation. We use a higher-order discontinuous Galerkin finite element method for discretization and motivate the numerical upwind scheme and its limitations from the continuous weak space--time formulation of the transport equation. We present a numerical implementation that builds on the finite element packages FEniCS and dolfin-adjoint. To demonstrate the efficacy of the proposed approach, numerical results for the registration of an input to a target MRI of two distinct individuals are presented. Moreover, it is shown that the registration transforms a manually crafted input mesh into a new mesh for the target subject whilst preserving mesh quality. Challenges of the algorithm are discussed.
△ Less
Submitted 9 April, 2024; v1 submitted 4 May, 2023;
originally announced May 2023.
-
The Proximal Map of the Weighted Mean Absolute Error
Authors:
Lukas Baumgärtner,
Roland Herzog,
Stephan Schmidt,
Manuel Weiß
Abstract:
We investigate the proximal map for the weighted mean absolute error function. An algorithm for its efficient and vectorized evaluation is presented. As a demonstration, this algorithm is applied as part of a checkerboard algorithm to solve a total-variation image denoising (ROF) problem as well as a non-smooth energy minimization problem.
We investigate the proximal map for the weighted mean absolute error function. An algorithm for its efficient and vectorized evaluation is presented. As a demonstration, this algorithm is applied as part of a checkerboard algorithm to solve a total-variation image denoising (ROF) problem as well as a non-smooth energy minimization problem.
△ Less
Submitted 8 February, 2023; v1 submitted 27 September, 2022;
originally announced September 2022.
-
Total Generalized Variation for Piecewise Constant Functions on Triangular Meshes with Applications in Imaging
Authors:
Lukas Baumgärtner,
Ronny Bergmann,
Roland Herzog,
Stephan Schmidt,
José Vidal-Núñez
Abstract:
We propose a novel discrete concept for the total generalized variation (TGV), which has originally been derived to reduce the staircasing effect in classical total variation (TV) regularization, in image denoising problems. We describe discrete, second-order TGV for piecewise constant functions on triangular meshes, thus allowing the TGV functional to be applied to more general data structures th…
▽ More
We propose a novel discrete concept for the total generalized variation (TGV), which has originally been derived to reduce the staircasing effect in classical total variation (TV) regularization, in image denoising problems. We describe discrete, second-order TGV for piecewise constant functions on triangular meshes, thus allowing the TGV functional to be applied to more general data structures than pixel images, and in particular in the context of finite element discretizations. Particular attention is given to the description of the kernel of the TGV functional, which, in the continuous setting, consists of linear polynomials. We discuss how to take advantage of this kernel structure using piecewise constant functions on triangular meshes. Numerical experiments include denoising and inpainting problems for images defined on non-standard grids, including data from a 3D scanner.
△ Less
Submitted 26 September, 2022; v1 submitted 24 June, 2022;
originally announced June 2022.
-
Mesh Denoising and Inpainting using the Total Variation of the Normal and a Shape Newton Approach
Authors:
Lukas Baumgärtner,
Ronny Bergmann,
Roland Herzog,
Stephan Schmidt,
José Vidal-Núñez,
Manuel Weiß
Abstract:
We present a novel approach to denoising and inpainting problems for surface meshes. The purpose of these problems is to remove noise or fill in missing parts while preserving important features such as sharp edges. A discrete variant of the total variation of the unit normal vector field serves as a regularizing functional to achieve these goals. In order to solve the resulting problem, we use a…
▽ More
We present a novel approach to denoising and inpainting problems for surface meshes. The purpose of these problems is to remove noise or fill in missing parts while preserving important features such as sharp edges. A discrete variant of the total variation of the unit normal vector field serves as a regularizing functional to achieve these goals. In order to solve the resulting problem, we use a version of the split Bregman (ADMM) iteration adapted to the problem. A new formulation of the total variation regularizer, as well as the use of an inexact Newton method for the shape optimization step, bring significant speed-up compared to earlier methods. Numerical examples are included, demonstrating the performance of our algorithm with some complex 3D geometries.
△ Less
Submitted 12 March, 2024; v1 submitted 21 December, 2020;
originally announced December 2020.
