-
Directional differentiability for solution operators of sweeping processes with convex polyhedral admissible sets
Authors:
Martin Brokate,
Constantin Christof
Abstract:
We study directional differentiability properties of solution operators of rate-independent evolution variational inequalities with full-dimensional convex polyhedral admissible sets. It is shown that, if the space of continuous functions of bounded variation is used as the domain of definition, then the most prototypical examples of such solution operators - the vector play and stop - are Hadamar…
▽ More
We study directional differentiability properties of solution operators of rate-independent evolution variational inequalities with full-dimensional convex polyhedral admissible sets. It is shown that, if the space of continuous functions of bounded variation is used as the domain of definition, then the most prototypical examples of such solution operators - the vector play and stop - are Hadamard directionally differentiable in a pointwise manner if and only if the admissible set is non-obtuse. We further prove that, in those cases where they exist, the directional derivatives of the vector play and stop are uniquely characterized by a system of projection identities and variational inequalities and that directional differentiability cannot be expected in the obtuse case even if the solution operator is restricted to the space of Lipschitz continuous functions. Our results can be used, for example, to formulate Bouligand stationarity conditions for optimal control problems involving sweeping processes.
△ Less
Submitted 22 March, 2025;
originally announced March 2025.
-
A Globalized Inexact Semismooth Newton Method for Nonsmooth Fixed-point Equations involving Variational Inequalities
Authors:
Amal Alphonse,
Constantin Christof,
Michael Hintermüller,
Ioannis P. A. Papadopoulos
Abstract:
We develop a semismooth Newton framework for the numerical solution of fixed-point equations that are posed in Banach spaces. The framework is motivated by applications in the field of obstacle-type quasi-variational inequalities and implicit obstacle problems. It is discussed in a general functional analytic setting and allows for inexact function evaluations and Newton steps. Moreover, if a cert…
▽ More
We develop a semismooth Newton framework for the numerical solution of fixed-point equations that are posed in Banach spaces. The framework is motivated by applications in the field of obstacle-type quasi-variational inequalities and implicit obstacle problems. It is discussed in a general functional analytic setting and allows for inexact function evaluations and Newton steps. Moreover, if a certain contraction assumption holds, we show that it is possible to globalize the algorithm by means of the Banach fixed-point theorem and to ensure $q$-superlinear convergence to the problem solution for arbitrary starting values. By means of a localization technique, our Newton method can also be used to determine solutions of fixed-point equations that are only locally contractive and not uniquely solvable. We apply our algorithm to a quasi-variational inequality which arises in thermoforming and which not only involves the obstacle problem as a source of nonsmoothness but also a semilinear PDE containing a nondifferentiable Nemytskii operator. Our analysis is accompanied by numerical experiments that illustrate the mesh-independence and $q$-superlinear convergence of the developed solution algorithm.
△ Less
Submitted 29 September, 2024;
originally announced September 2024.
-
Optimal Control of Semilinear Elliptic Partial Differential Equations with Non-Lipschitzian Nonlinearities
Authors:
Constantin Christof
Abstract:
We study optimal control problems that are governed by semilinear elliptic partial differential equations that involve non-Lipschitzian nonlinearities. It is shown that, for a certain class of such PDEs, the solution map is Fréchet differentiable even though the differential operator contains a nondifferentiable term. We exploit this effect to establish first-order necessary optimality conditions…
▽ More
We study optimal control problems that are governed by semilinear elliptic partial differential equations that involve non-Lipschitzian nonlinearities. It is shown that, for a certain class of such PDEs, the solution map is Fréchet differentiable even though the differential operator contains a nondifferentiable term. We exploit this effect to establish first-order necessary optimality conditions for minimizers of the considered control problems. The resulting KKT-conditions take the form of coupled PDE-systems that are posed in non-Muckenhoupt weighted Sobolev spaces and raise interesting questions regarding the regularity of optimal controls, the derivation of second-order optimality conditions, and the analysis of finite element discretizations.
△ Less
Submitted 30 November, 2024; v1 submitted 5 June, 2024;
originally announced June 2024.
-
Gas Source Localization Using physics Guided Neural Networks
Authors:
Victor Scott Prieto Ruiz,
Patrick Hinsen,
Thomas Wiedemann,
Constantin Christof,
Dmitriy Shutin
Abstract:
This work discusses a novel method for estimating the location of a gas source based on spatially distributed concentration measurements taken, e.g., by a mobile robot or flying platform that follows a predefined trajectory to collect samples. The proposed approach uses a Physics-Guided Neural Network to approximate the gas dispersion with the source location as an additional network input. After…
▽ More
This work discusses a novel method for estimating the location of a gas source based on spatially distributed concentration measurements taken, e.g., by a mobile robot or flying platform that follows a predefined trajectory to collect samples. The proposed approach uses a Physics-Guided Neural Network to approximate the gas dispersion with the source location as an additional network input. After an initial offline training phase, the neural network can be used to efficiently solve the inverse problem of localizing the gas source based on measurements. The proposed approach allows avoiding rather costly numerical simulations of gas physics needed for solving inverse problems. Our experiments show that the method localizes the source well, even when dealing with measurements affected by noise.
△ Less
Submitted 7 May, 2024;
originally announced May 2024.
-
Energy Space Newton Differentiability for Solution Maps of Unilateral and Bilateral Obstacle Problems
Authors:
Constantin Christof,
Gerd Wachsmuth
Abstract:
We prove that the solution operator of the classical unilateral obstacle problem on a nonempty open bounded set $Ω\subset \mathbb{R}^d$, $d \in \mathbb{N}$, is Newton differentiable as a function from $L^p(Ω)$ to $H_0^1(Ω)$ whenever $\max(1, 2d/(d+2)) < p \leq \infty$. By exploiting this Newton differentiability property, results on angled subspaces in $H^{-1}(Ω)$, and a formula for orthogonal pro…
▽ More
We prove that the solution operator of the classical unilateral obstacle problem on a nonempty open bounded set $Ω\subset \mathbb{R}^d$, $d \in \mathbb{N}$, is Newton differentiable as a function from $L^p(Ω)$ to $H_0^1(Ω)$ whenever $\max(1, 2d/(d+2)) < p \leq \infty$. By exploiting this Newton differentiability property, results on angled subspaces in $H^{-1}(Ω)$, and a formula for orthogonal projections onto direct sums, we further show that the solution map of the classical bilateral obstacle problem is Newton differentiable as a function from $L^p(Ω)$ to $H_0^1(Ω)\cap L^q(Ω)$ whenever $\max(1, d/2) < p \leq \infty$ and $1 \leq q <\infty$. For both the unilateral and the bilateral case, we provide explicit formulas for the Newton derivative. As a concrete application example for our results, we consider the numerical solution of an optimal control problem with $H_0^1(Ω)$-controls and box-constraints by means of a semismooth Newton method.
△ Less
Submitted 29 August, 2023;
originally announced August 2023.
-
On the Identification and Optimization of Nonsmooth Superposition Operators in Semilinear Elliptic PDEs
Authors:
Constantin Christof,
Julia Kowalczyk
Abstract:
We study an infinite-dimensional optimization problem that aims to identify the Nemytskii operator in the nonlinear part of a prototypical semilinear elliptic partial differential equation (PDE) which minimizes the distance between the PDE-solution and a given desired state. In contrast to previous works, we consider this identification problem in a low-regularity regime in which the function indu…
▽ More
We study an infinite-dimensional optimization problem that aims to identify the Nemytskii operator in the nonlinear part of a prototypical semilinear elliptic partial differential equation (PDE) which minimizes the distance between the PDE-solution and a given desired state. In contrast to previous works, we consider this identification problem in a low-regularity regime in which the function inducing the Nemytskii operator is a-priori only known to be an element of $H^1_{loc}(\mathbb{R})$. This makes the studied problem class a suitable point of departure for the rigorous analysis of training problems for learning-informed PDEs in which an unknown superposition operator is approximated by means of a neural network with nonsmooth activation functions (ReLU, leaky-ReLU, etc.). We establish that, despite the low regularity of the controls, it is possible to derive a classical stationarity system for local minimizers and to solve the considered problem by means of a gradient projection method. The convergence of the resulting algorithm is proven in the function space setting. It is also shown that the established first-order necessary optimality conditions imply that locally optimal superposition operators share various characteristic properties with commonly used activation functions: They are always sigmoidal, continuously differentiable away from the origin, and typically possess a distinct kink at zero. The paper concludes with numerical experiments which confirm the theoretical findings.
△ Less
Submitted 2 February, 2024; v1 submitted 8 June, 2023;
originally announced June 2023.
-
Strong Stationarity Conditions for Optimal Control Problems Governed by a Rate-Independent Evolution Variational Inequality
Authors:
Martin Brokate,
Constantin Christof
Abstract:
We prove strong stationarity conditions for optimal control problems that are governed by a prototypical rate-independent evolution variational inequality, i.e., first-order necessary optimality conditions in the form of a primal-dual multiplier system that are equivalent to the purely primal notion of Bouligand stationarity. Our analysis relies on recent results on the Hadamard directional differ…
▽ More
We prove strong stationarity conditions for optimal control problems that are governed by a prototypical rate-independent evolution variational inequality, i.e., first-order necessary optimality conditions in the form of a primal-dual multiplier system that are equivalent to the purely primal notion of Bouligand stationarity. Our analysis relies on recent results on the Hadamard directional differentiability of the scalar stop operator and a new concept of temporal polyhedricity that generalizes classical ideas of Mignot. The established strong stationarity system is compared with known optimality conditions for optimal control problems governed by elliptic obstacle-type variational inequalities and stationarity systems obtained by regularization.
△ Less
Submitted 18 July, 2023; v1 submitted 2 May, 2022;
originally announced May 2022.
-
On the Omnipresence of Spurious Local Minima in Certain Neural Network Training Problems
Authors:
Constantin Christof,
Julia Kowalczyk
Abstract:
We study the loss landscape of training problems for deep artificial neural networks with a one-dimensional real output whose activation functions contain an affine segment and whose hidden layers have width at least two. It is shown that such problems possess a continuum of spurious (i.e., not globally optimal) local minima for all target functions that are not affine. In contrast to previous wor…
▽ More
We study the loss landscape of training problems for deep artificial neural networks with a one-dimensional real output whose activation functions contain an affine segment and whose hidden layers have width at least two. It is shown that such problems possess a continuum of spurious (i.e., not globally optimal) local minima for all target functions that are not affine. In contrast to previous works, our analysis covers all sampling and parameterization regimes, general differentiable loss functions, arbitrary continuous nonpolynomial activation functions, and both the finite- and infinite-dimensional setting. It is further shown that the appearance of the spurious local minima in the considered training problems is a direct consequence of the universal approximation theorem and that the underlying mechanisms also cause, e.g., $L^p$-best approximation problems to be ill-posed in the sense of Hadamard for all networks that do not have a dense image. The latter result also holds without the assumption of local affine linearity and without any conditions on the hidden layers.
△ Less
Submitted 15 June, 2023; v1 submitted 23 February, 2022;
originally announced February 2022.
-
Semismoothness for Solution Operators of Obstacle-Type Variational Inequalities with Applications in Optimal Control
Authors:
Constantin Christof,
Gerd Wachsmuth
Abstract:
We prove that solution operators of elliptic obstacle-type variational inequalities (or, more generally, locally Lipschitz continuous functions possessing certain pointwise-a.e. convexity properties) are Newton differentiable when considered as maps between suitable Lebesgue spaces and equipped with the strong-weak Bouligand differential as a generalized set-valued derivative. It is shown that thi…
▽ More
We prove that solution operators of elliptic obstacle-type variational inequalities (or, more generally, locally Lipschitz continuous functions possessing certain pointwise-a.e. convexity properties) are Newton differentiable when considered as maps between suitable Lebesgue spaces and equipped with the strong-weak Bouligand differential as a generalized set-valued derivative. It is shown that this Newton differentiability allows to solve optimal control problems with H1-cost terms and one-sided pointwise control constraints by means of a semismooth Newton method. The superlinear convergence of the resulting algorithm is proved in the infinite-dimensional setting and its mesh independence is demonstrated in numerical experiments. We expect that the findings of this paper are also helpful for the design of numerical solution procedures for quasi-variational inequalities and the optimal control of obstacle-type variational problems.
△ Less
Submitted 8 June, 2023; v1 submitted 22 December, 2021;
originally announced December 2021.
-
Lipschitz Stability and Hadamard Directional Differentiability for Elliptic and Parabolic Obstacle-Type Quasi-Variational Inequalities
Authors:
Constantin Christof,
Gerd Wachsmuth
Abstract:
This paper is concerned with the sensitivity analysis of a class of parameterized fixed-point problems that arise in the context of obstacle-type quasi-variational inequalities. We prove that, if the operators in the considered fixed-point equation satisfy a positive superhomogeneity condition, then the maximal and minimal element of the solution set of the problem depend locally Lipschitz continu…
▽ More
This paper is concerned with the sensitivity analysis of a class of parameterized fixed-point problems that arise in the context of obstacle-type quasi-variational inequalities. We prove that, if the operators in the considered fixed-point equation satisfy a positive superhomogeneity condition, then the maximal and minimal element of the solution set of the problem depend locally Lipschitz continuously on the involved parameters. We further show that, if certain concavity conditions hold, then the maximal solution mapping is Hadamard directionally differentiable and its directional derivatives are precisely the minimal solutions of suitably defined linearized fixed-point equations. In contrast to prior results, our analysis requires neither a Dirichlet space structure, nor restrictive assumptions on the mapping behavior and regularity of the involved operators, nor sign conditions on the directions that are considered in the directional derivatives. Our approach further covers the elliptic and parabolic setting simultaneously and also yields Hadamard directional differentiability results in situations in which the solution set of the fixed-point equation is a continuum and a characterization of directional derivatives via linearized auxiliary problems is provably impossible. To illustrate that our results can be used to study interesting problems arising in practice, we apply them to establish the Hadamard directional differentiability of the solution operator of a nonlinear elliptic quasi-variational inequality, which emerges in impulse control and in which the obstacle mapping is obtained by taking essential infima over certain parts of the underlying domain, and of the solution mapping of a parabolic quasi-variational inequality, which involves boundary controls and in which the state-to-obstacle relationship is described by a partial differential equation.
△ Less
Submitted 12 May, 2021;
originally announced May 2021.
-
On the Stability Properties and the Optimization Landscape of Training Problems with Squared Loss for Neural Networks and General Nonlinear Conic Approximation Schemes
Authors:
Constantin Christof
Abstract:
We study the optimization landscape and the stability properties of training problems with squared loss for neural networks and general nonlinear conic approximation schemes. It is demonstrated that, if a nonlinear conic approximation scheme is considered that is (in an appropriately defined sense) more expressive than a classical linear approximation approach and if there exist unrealizable label…
▽ More
We study the optimization landscape and the stability properties of training problems with squared loss for neural networks and general nonlinear conic approximation schemes. It is demonstrated that, if a nonlinear conic approximation scheme is considered that is (in an appropriately defined sense) more expressive than a classical linear approximation approach and if there exist unrealizable label vectors, then a training problem with squared loss is necessarily unstable in the sense that its solution set depends discontinuously on the label vector in the training data. We further prove that the same effects that are responsible for these instability properties are also the reason for the emergence of saddle points and spurious local minima, which may be arbitrarily far away from global solutions, and that neither the instability of the training problem nor the existence of spurious local minima can, in general, be overcome by adding a regularization term to the objective function that penalizes the size of the parameters in the approximation scheme. The latter results are shown to be true regardless of whether the assumption of realizability is satisfied or not. We demonstrate that our analysis in particular applies to training problems for free-knot interpolation schemes and deep and shallow neural networks with variable widths that involve an arbitrary mixture of various activation functions (e.g., binary, sigmoid, tanh, arctan, soft-sign, ISRU, soft-clip, SQNL, ReLU, leaky ReLU, soft-plus, bent identity, SILU, ISRLU, and ELU). In summary, the findings of this paper illustrate that the improved approximation properties of neural networks and general nonlinear conic approximation instruments are linked in a direct and quantifiable way to undesirable properties of the optimization problems that have to be solved in order to train them.
△ Less
Submitted 2 December, 2021; v1 submitted 6 November, 2020;
originally announced November 2020.
-
On the Nonuniqueness and Instability of Solutions of Tracking-Type Optimal Control Problems
Authors:
Constantin Christof,
Dominik Hafemeyer
Abstract:
We study tracking-type optimal control problems that involve a non-affine, weak-to-weak continuous control-to-state mapping, a desired state $y_d$, and a desired control $u_d$. It is proved that such problems are always nonuniquely solvable for certain choices of the tuple $(y_d, u_d)$ and instable in the sense that the set of solutions (interpreted as a multivalued function of $(y_d, u_d)$) does…
▽ More
We study tracking-type optimal control problems that involve a non-affine, weak-to-weak continuous control-to-state mapping, a desired state $y_d$, and a desired control $u_d$. It is proved that such problems are always nonuniquely solvable for certain choices of the tuple $(y_d, u_d)$ and instable in the sense that the set of solutions (interpreted as a multivalued function of $(y_d, u_d)$) does not admit a continuous selection.
△ Less
Submitted 3 February, 2021; v1 submitted 16 July, 2020;
originally announced July 2020.
-
On Second-Order Optimality Conditions for Optimal Control Problems Governed by the Obstacle Problem
Authors:
Constantin Christof,
Gerd Wachsmuth
Abstract:
This paper is concerned with second-order optimality conditions for Tikhonov regularized optimal control problems governed by the obstacle problem. Using a simple observation that allows to characterize the structure of optimal controls on the active set, we derive various conditions that guarantee the local/global optimality of first-order stationary points and/or the local/global quadratic growt…
▽ More
This paper is concerned with second-order optimality conditions for Tikhonov regularized optimal control problems governed by the obstacle problem. Using a simple observation that allows to characterize the structure of optimal controls on the active set, we derive various conditions that guarantee the local/global optimality of first-order stationary points and/or the local/global quadratic growth of the reduced objective function. Our analysis extends and refines existing results from the literature, and also covers those situations where the problem at hand involves additional box-constraints on the control. As a byproduct, our approach shows in particular that Tikhonov regularized optimal control problems for the obstacle problem can be reformulated as state-constrained optimal control problems for the Poisson equation, and that problems involving a subharmonic obstacle and a convex objective function are uniquely solvable. The paper concludes with three counterexamples which illustrate that rather peculiar effects can occur in the analysis of second-order optimality conditions for optimal control problems governed by the obstacle problem, and that necessary second-order conditions for such problems may be hard to derive.
△ Less
Submitted 21 June, 2019;
originally announced June 2019.
-
A non-smooth trust-region method for locally Lipschitz functions with application to optimization problems constrained by variational inequalities
Authors:
Constantin Christof,
Juan Carlos De Los Reyes,
Christian Meyer
Abstract:
We propose a nonsmooth trust-region method for solving optimization problems with locally Lipschitz continuous functions, with application to problems constrained by variational inequalities of the second kind. Under suitable assumptions on the model functions, convergence of the general algorithm to a C-stationary point is verified. For variational inequality constrained problems, we are able to…
▽ More
We propose a nonsmooth trust-region method for solving optimization problems with locally Lipschitz continuous functions, with application to problems constrained by variational inequalities of the second kind. Under suitable assumptions on the model functions, convergence of the general algorithm to a C-stationary point is verified. For variational inequality constrained problems, we are able to properly characterize the Bouligand subdifferential of the reduced cost function and, based on that, we propose a computable trust-region model which fulfills the convergence hypotheses of the general algorithm. The article concludes with the experimental study of the main properties of the proposed method based on two different numerical instances.
△ Less
Submitted 16 January, 2018; v1 submitted 8 November, 2017;
originally announced November 2017.
-
Differential Sensitivity Analysis of Variational Inequalities with Locally Lipschitz Continuous Solution Operators
Authors:
Constantin Christof,
Gerd Wachsmuth
Abstract:
This paper is concerned with the differential sensitivity analysis of variational inequalities in Banach spaces whose solution operators satisfy a generalized Lipschitz condition. We prove a sufficient criterion for the directional differentiability of the solution map that turns out to be also necessary for elliptic variational inequalities in Hilbert spaces (even in the presence of asymmetric bi…
▽ More
This paper is concerned with the differential sensitivity analysis of variational inequalities in Banach spaces whose solution operators satisfy a generalized Lipschitz condition. We prove a sufficient criterion for the directional differentiability of the solution map that turns out to be also necessary for elliptic variational inequalities in Hilbert spaces (even in the presence of asymmetric bilinear forms, nonlinear operators and nonconvex functionals). In contrast to classical results, our method of proof does not rely on Attouch's theorem on the characterization of Mosco convergence but is fully elementary. Moreover, our technique allows us to also study those cases where the variational inequality at hand is not uniquely solvable and where directional differentiability can only be obtained w.r.t. the weak or the weak-$\star$ topology of the underlying space. As tangible examples, we consider a variational inequality arising in elastoplasticity, the projection onto prox-regular sets, and a bang-bang optimal control problem.
△ Less
Submitted 7 November, 2017;
originally announced November 2017.
-
On the Non-Polyhedricity of Sets with Upper and Lower Bounds in Dual Spaces
Authors:
Constantin Christof,
Gerd Wachsmuth
Abstract:
We demonstrate that the set $L^\infty(X, [-1,1])$ of all measurable functions over a Borel measure space $(X, \mathcal B, μ)$ with values in the unit interval is typically non-polyhedric when interpreted as a subset of a dual space. Our findings contrast the classical result that subsets of Dirichlet spaces with pointwise upper and lower bounds are polyhedric. In particular, additional structural…
▽ More
We demonstrate that the set $L^\infty(X, [-1,1])$ of all measurable functions over a Borel measure space $(X, \mathcal B, μ)$ with values in the unit interval is typically non-polyhedric when interpreted as a subset of a dual space. Our findings contrast the classical result that subsets of Dirichlet spaces with pointwise upper and lower bounds are polyhedric. In particular, additional structural assumptions are unavoidable when the concept of polyhedricity is used to study the differentiability properties of solution maps to variational inequalities of the second kind in, e.g., the spaces $H^{1/2}(\partial Ω)$ or $H_0^1(Ω)$.
△ Less
Submitted 7 November, 2017;
originally announced November 2017.
-
No-Gap Second-Order Conditions via a Directional Curvature Functional
Authors:
Constantin Christof,
Gerd Wachsmuth
Abstract:
This paper is concerned with necessary and sufficient second-order conditions for finite-dimensional and infinite-dimensional constrained optimization problems. Using a suitably defined directional curvature functional for the admissible set, we derive no-gap second-order optimality conditions in an abstract functional analytic setting. Our theory not only covers those cases where the classical as…
▽ More
This paper is concerned with necessary and sufficient second-order conditions for finite-dimensional and infinite-dimensional constrained optimization problems. Using a suitably defined directional curvature functional for the admissible set, we derive no-gap second-order optimality conditions in an abstract functional analytic setting. Our theory not only covers those cases where the classical assumptions of polyhedricity or second-order regularity are satisfied but also allows to study problems in the absence of these requirements. As a tangible example, we consider no-gap second-order conditions for bang-bang optimal control problems.
△ Less
Submitted 25 January, 2021; v1 submitted 24 July, 2017;
originally announced July 2017.
-
Optimal control of a non-smooth semilinear elliptic equation
Authors:
Constantin Christof,
Christian Clason,
Christian Meyer,
Stephan Walther
Abstract:
This paper is concerned with an optimal control problem governed by a non-smooth semilinear elliptic equation. We show that the control-to-state mapping is directionally differentiable and precisely characterize its Bouligand subdifferential. By means of a suitable regularization, first-order optimality conditions including an adjoint equation are derived and afterwards interpreted in light of the…
▽ More
This paper is concerned with an optimal control problem governed by a non-smooth semilinear elliptic equation. We show that the control-to-state mapping is directionally differentiable and precisely characterize its Bouligand subdifferential. By means of a suitable regularization, first-order optimality conditions including an adjoint equation are derived and afterwards interpreted in light of the previously obtained characterization. In addition, the directional derivative of the control-to-state mapping is used to establish strong stationarity conditions. While the latter conditions are shown to be stronger, we demonstrate by numerical examples that the former conditions are amenable to numerical solution using a semi-smooth Newton method.
△ Less
Submitted 27 November, 2017; v1 submitted 2 May, 2017;
originally announced May 2017.
