-
Adjoint Monte Carlo Method
Authors:
Russel Caflisch,
Yunan Yang
Abstract:
This survey explores the development of adjoint Monte Carlo methods for solving optimization problems governed by kinetic equations, a common challenge in areas such as plasma control and device design. These optimization problems are particularly demanding due to the high dimensionality of the phase space and the randomness in evaluating the objective functional, a consequence of using a forward…
▽ More
This survey explores the development of adjoint Monte Carlo methods for solving optimization problems governed by kinetic equations, a common challenge in areas such as plasma control and device design. These optimization problems are particularly demanding due to the high dimensionality of the phase space and the randomness in evaluating the objective functional, a consequence of using a forward Monte Carlo solver. To overcome these difficulties, a range of ``adjoint Monte Carlo methods'' have been devised. These methods skillfully combine Monte Carlo gradient estimators with PDE-constrained optimization, introducing innovative solutions tailored for kinetic applications. In this review, we begin by examining three primary strategies for Monte Carlo gradient estimation: the score function approach, the reparameterization trick, and the coupling method. We also delve into the adjoint-state method, an essential element in PDE-constrained optimization. Focusing on applications in the radiative transfer equation and the nonlinear Boltzmann equation, we provide a comprehensive guide on how to integrate Monte Carlo gradient techniques within both the optimize-then-discretize and the discretize-then-optimize frameworks from PDE-constrained optimization. This approach leads to the formulation of effective adjoint Monte Carlo methods, enabling efficient gradient estimation in complex, high-dimensional optimization problems.
△ Less
Submitted 22 May, 2024; v1 submitted 16 January, 2024;
originally announced January 2024.
-
Gradient-based Monte Carlo methods for relaxation approximations of hyperbolic conservation laws
Authors:
Giulia Bertaglia,
Lorenzo Pareschi,
Russel E. Caflisch
Abstract:
Particle methods based on evolving the spatial derivatives of the solution were originally introduced to simulate reaction-diffusion processes, inspired by vortex methods for the Navier--Stokes equations. Such methods, referred to as gradient random walk methods, were extensively studied in the '90s and have several interesting features, such as being grid free, automatically adapting to the solut…
▽ More
Particle methods based on evolving the spatial derivatives of the solution were originally introduced to simulate reaction-diffusion processes, inspired by vortex methods for the Navier--Stokes equations. Such methods, referred to as gradient random walk methods, were extensively studied in the '90s and have several interesting features, such as being grid free, automatically adapting to the solution by concentrating elements where the gradient is large and significantly reducing the variance of the standard random walk approach. In this work, we revive these ideas by showing how to generalize the approach to a larger class of partial differential equations, including hyperbolic systems of conservation laws. To achieve this goal, we first extend the classical Monte Carlo method to relaxation approximation of systems of conservation laws, and subsequently consider a novel particle dynamics based on the spatial derivatives of the solution. The methodology, combined with asymptotic-preserving splitting discretization, yields a way to construct a new class of gradient-based Monte Carlo methods for hyperbolic systems of conservation laws. Several results in one spatial dimension for scalar equations and systems of conservation laws show that the new methods are very promising and yield remarkable improvements compared to standard Monte Carlo approaches, either in terms of variance reduction as well as in describing the shock structure.
△ Less
Submitted 5 August, 2023;
originally announced August 2023.
-
Adjoint DSMC for Nonlinear Spatially-Homogeneous Boltzmann Equation With a General Collision Model
Authors:
Yunan Yang,
Denis Silantyev,
Russel Caflisch
Abstract:
We derive an adjoint method for the Direct Simulation Monte Carlo (DSMC) method for the spatially homogeneous Boltzmann equation with a general collision law. This generalizes our previous results in [Caflisch, R., Silantyev, D. and Yang, Y., 2021. Journal of Computational Physics, 439, p.110404], which was restricted to the case of Maxwell molecules, for which the collision rate is constant. The…
▽ More
We derive an adjoint method for the Direct Simulation Monte Carlo (DSMC) method for the spatially homogeneous Boltzmann equation with a general collision law. This generalizes our previous results in [Caflisch, R., Silantyev, D. and Yang, Y., 2021. Journal of Computational Physics, 439, p.110404], which was restricted to the case of Maxwell molecules, for which the collision rate is constant. The main difficulty in generalizing the previous results is that a rejection sampling step is required in the DSMC algorithm in order to handle the variable collision rate. We find a new term corresponding to the so-called score function in the adjoint equation and a new adjoint Jacobian matrix capturing the dependence of the collision parameter on the velocities. The new formula works for a much more general class of collision models.
△ Less
Submitted 30 July, 2022; v1 submitted 23 July, 2022;
originally announced July 2022.
-
Adjoint DSMC for nonlinear Boltzmann equation constrained optimization
Authors:
Russel Caflisch,
Denis Silantyev,
Yunan Yang
Abstract:
Applications for kinetic equations such as optimal design and inverse problems often involve finding unknown parameters through gradient-based optimization algorithms. Based on the adjoint-state method, we derive two different frameworks for approximating the gradient of an objective functional constrained by the nonlinear Boltzmann equation. While the forward problem can be solved by the DSMC met…
▽ More
Applications for kinetic equations such as optimal design and inverse problems often involve finding unknown parameters through gradient-based optimization algorithms. Based on the adjoint-state method, we derive two different frameworks for approximating the gradient of an objective functional constrained by the nonlinear Boltzmann equation. While the forward problem can be solved by the DSMC method, it is difficult to efficiently solve the high-dimensional continuous adjoint equation obtained by the "optimize-then-discretize" approach. This challenge motivates us to propose an adjoint DSMC method following the "discretize-then-optimize" approach for Boltzmann-constrained optimization. We also analyze the properties of the two frameworks and their connections. Several numerical examples are presented to demonstrate their accuracy and efficiency.
△ Less
Submitted 10 May, 2021; v1 submitted 2 September, 2020;
originally announced September 2020.
-
Complex singularities and PDEs
Authors:
R. E. Caflisch,
F. Gargano,
M. Sammartino,
V. Sciacca
Abstract:
In this paper we give a review on the computational methods used to characterize the complex singularities developed by some relevant PDEs. We begin by reviewing the singularity tracking method based on the analysis of the Fourier spectrum. We then introduce other methods generally used to detect the hidden singularities. In particular we show some applications of the Padé approximation, of the Ki…
▽ More
In this paper we give a review on the computational methods used to characterize the complex singularities developed by some relevant PDEs. We begin by reviewing the singularity tracking method based on the analysis of the Fourier spectrum. We then introduce other methods generally used to detect the hidden singularities. In particular we show some applications of the Padé approximation, of the Kida method, and of Borel-Polya method. We apply these techniques to the study of the singularity formation of some nonlinear dispersive and dissipative one dimensional PDE of the 2D Prandtl equation, of the 2D KP equation, and to Navier-Stokes equation for high Reynolds number incompressible flows in the case of interaction with rigid boundaries.
△ Less
Submitted 25 November, 2015;
originally announced December 2015.
-
Compressed Wannier modes found from an $L_1$ regularized energy functional
Authors:
Farzin Barekat,
Ke Yin,
Russel E. Caflisch,
Stanley J. Osher,
Rongjie Lai,
Vidvuds Ozolins
Abstract:
We propose a method for calculating Wannier functions of periodic solids directly from a modified variational principle for the energy, subject to the requirement that the Wannier functions are orthogonal to all their translations ("shift-orthogonality"). Localization is achieved by adding an $L_1$ regularization term to the energy functional. This approach results in "compressed" Wannier modes wi…
▽ More
We propose a method for calculating Wannier functions of periodic solids directly from a modified variational principle for the energy, subject to the requirement that the Wannier functions are orthogonal to all their translations ("shift-orthogonality"). Localization is achieved by adding an $L_1$ regularization term to the energy functional. This approach results in "compressed" Wannier modes with compact support, where one parameter $μ$ controls the trade-off between the accuracy of the total energy and the size of the support of the Wannier modes. Efficient algorithms for shift-orthogonalization and solution of the variational minimization problem are demonstrated.
△ Less
Submitted 26 March, 2014;
originally announced March 2014.
-
Projection to the Set of Shift Orthogonal Functions
Authors:
Farzin Barekat,
Rongjie Lai,
Ke Yin,
Stanley Osher,
Russel Caflisch,
Vidvuds Ozolins
Abstract:
This paper presents a fast algorithm for projecting a given function to the set of shift orthogonal functions (i.e. set containing functions with unit $L^2$ norm that are orthogonal to their prescribed shifts). The algorithm can be parallelized easily and its computational complexity is bounded by $O(M\log(M))$, where $M$ is the number of coefficients used for storing the input. To derive the algo…
▽ More
This paper presents a fast algorithm for projecting a given function to the set of shift orthogonal functions (i.e. set containing functions with unit $L^2$ norm that are orthogonal to their prescribed shifts). The algorithm can be parallelized easily and its computational complexity is bounded by $O(M\log(M))$, where $M$ is the number of coefficients used for storing the input. To derive the algorithm, a particular class of basis called Shift Orthogonal Basis Functions are introduced and some theory regarding them is developed.
△ Less
Submitted 20 February, 2014;
originally announced February 2014.
-
PDEs with Compressed Solutions
Authors:
Russel E. Caflisch,
Stanley J. Osher,
Hayden Schaeffer,
Giang Tran
Abstract:
Sparsity plays a central role in recent developments in signal processing, linear algebra, statistics, optimization, and other fields. In these developments, sparsity is promoted through the addition of an $L^1$ norm (or related quantity) as a constraint or penalty in a variational principle. We apply this approach to partial differential equations that come from a variational quantity, either by…
▽ More
Sparsity plays a central role in recent developments in signal processing, linear algebra, statistics, optimization, and other fields. In these developments, sparsity is promoted through the addition of an $L^1$ norm (or related quantity) as a constraint or penalty in a variational principle. We apply this approach to partial differential equations that come from a variational quantity, either by minimization (to obtain an elliptic PDE) or by gradient flow (to obtain a parabolic PDE). Also, we show that some PDEs can be rewritten in an $L^1$ form, such as the divisible sandpile problem and signum-Gordon. Addition of an $L^1$ term in the variational principle leads to a modified PDE where a subgradient term appears. It is known that modified PDEs of this form will often have solutions with compact support, which corresponds to the discrete solution being sparse. We show that this is advantageous numerically through the use of efficient algorithms for solving $L^1$ based problems.
△ Less
Submitted 1 August, 2014; v1 submitted 22 November, 2013;
originally announced November 2013.
-
Simulation with Fluctuating and Singular Rates
Authors:
Farzin Barekat,
Russel Caflisch
Abstract:
In this paper we present a method to generate independent samples for a general random variable, either continuous or discrete. The algorithm is an extension of the acceptance-rejection method, and it is particularly useful for kinetic simulation in which the rates are fluctuating in time and have singular limits, as occurs for example in simulation of recombination interactions in a plasma. Altho…
▽ More
In this paper we present a method to generate independent samples for a general random variable, either continuous or discrete. The algorithm is an extension of the acceptance-rejection method, and it is particularly useful for kinetic simulation in which the rates are fluctuating in time and have singular limits, as occurs for example in simulation of recombination interactions in a plasma. Although it depends on some additional requirements, the new method is easy to implement and rejects less samples than the acceptance-rejection method.
△ Less
Submitted 16 October, 2013;
originally announced October 2013.
-
Multilevel Monte Carlo simulation of Coulomb collisions
Authors:
M. S. Rosin,
L. F. Ricketson,
A. M. Dimits,
R. E. Caflisch,
B. I. Cohen
Abstract:
We present a new, for plasma physics, highly efficient multilevel Monte Carlo numerical method for simulating Coulomb collisions. The method separates and optimally minimizes the finite-timestep and finite-sampling errors inherent in the Langevin representation of the Landau-Fokker-Planck equation. It does so by combining multiple solutions to the underlying equations with varying numbers of times…
▽ More
We present a new, for plasma physics, highly efficient multilevel Monte Carlo numerical method for simulating Coulomb collisions. The method separates and optimally minimizes the finite-timestep and finite-sampling errors inherent in the Langevin representation of the Landau-Fokker-Planck equation. It does so by combining multiple solutions to the underlying equations with varying numbers of timesteps. For a desired level of accuracy epsilon, the computational cost of the method is order(epsilon^{-2}) or order(epsilon^{-2} (\ln epsilon)^2), depending on the underlying discretization, Milstein or Euler-Maruyama respectively. This is to be contrasted with a cost of order(epsilon^{-3}) for direct simulation Monte Carlo or binary collision methods. We successfully demonstrate the method with a classic beam diffusion test case in 2D, making use of the Levy area approximation for the correlated Milstein cross terms, and generating a computational saving of a factor of 100 for epsilon = 10^{-5}. We discuss the importance of the method for problems in which collisions constitute the computational rate limiting step, and its limitations.
△ Less
Submitted 14 October, 2013;
originally announced October 2013.
-
Compressed Modes for Variational Problems in Mathematics and Physics
Authors:
Vidvuds Ozoliņš,
Rongjie Lai,
Russel Caflisch,
Stanley Osher
Abstract:
This paper describes a general formalism for obtaining localized solutions to a class of problems in mathematical physics, which can be recast as variational optimization problems. This class includes the important cases of Schrödinger's equation in quantum mechanics and electromagnetic equations for light propagation in photonic crystals. These ideas can also be applied to develop a spatially loc…
▽ More
This paper describes a general formalism for obtaining localized solutions to a class of problems in mathematical physics, which can be recast as variational optimization problems. This class includes the important cases of Schrödinger's equation in quantum mechanics and electromagnetic equations for light propagation in photonic crystals. These ideas can also be applied to develop a spatially localized basis that spans the eigenspace of a differential operator, for instance, the Laplace operator, generalizing the concept of plane waves to an orthogonal real-space basis with multi-resolution capabilities.
△ Less
Submitted 28 August, 2013; v1 submitted 8 August, 2013;
originally announced August 2013.
-
An entropy based thermalization scheme for hybrid simulations of Coulomb collisions
Authors:
L. F. Ricketson,
M. S. Rosin,
R. E. Caflisch,
Andris M. Dimits
Abstract:
We formulate and test a hybrid fluid-Monte Carlo scheme for the treatment of elastic collisions in gases and plasmas. While our primary focus and demonstrations of applicability are for moderately collisional plasmas, as described by the Landau-Fokker-Planck equation, the method is expected to be applicable also to collision processes described by the Boltzmann equation. This scheme is similar to…
▽ More
We formulate and test a hybrid fluid-Monte Carlo scheme for the treatment of elastic collisions in gases and plasmas. While our primary focus and demonstrations of applicability are for moderately collisional plasmas, as described by the Landau-Fokker-Planck equation, the method is expected to be applicable also to collision processes described by the Boltzmann equation. This scheme is similar to the previously discussed velocity-based scheme [R. Caflisch, et. al, Multiscale Modeling & Simulation 7, 865, (2008)] and the scattering-angle-based scheme [A.M. Dimits, et. al, Bull. APS 55, no. 15 (2010, Abstract: XP9.00006)], but with a firmer theoretical basis and without the inherent limitation to the Landau-Fokker-Planck case. It gives a significant performance improvement (e.g., error for a given computational effort) over the velocity-based scheme. These features are achieved by assigning passive scalars to each simulated particle and tracking their evolution through collisions. The method permits a detailed error analysis that is confirmed by numerical results. The tests performed are for the evolution from anisotropic Maxwellian and a bump-on-tail distribution.
△ Less
Submitted 23 January, 2013;
originally announced January 2013.
-
Sparse Dynamics for Partial Differential Equations
Authors:
Hayden Schaeffer,
Stanley Osher,
Russel Caflisch,
Cory Hauck
Abstract:
We investigate the approximate dynamics of several differential equations when the solutions are restricted to a sparse subset of a given basis. The restriction is enforced at every time step by simply applying soft thresholding to the coefficients of the basis approximation. By reducing or compressing the information needed to represent the solution at every step, only the essential dynamics are…
▽ More
We investigate the approximate dynamics of several differential equations when the solutions are restricted to a sparse subset of a given basis. The restriction is enforced at every time step by simply applying soft thresholding to the coefficients of the basis approximation. By reducing or compressing the information needed to represent the solution at every step, only the essential dynamics are represented. In many cases, there are natural bases derived from the differential equations which promote sparsity. We find that our method successfully reduces the dynamics of convection equations, diffusion equations, weak shocks, and vorticity equations with high frequency source terms.
△ Less
Submitted 17 December, 2012;
originally announced December 2012.
-
European Option Pricing with Transaction Costs and Stochastic Volatility: an Asymptotic Analysis
Authors:
R. E. Caflisch,
G. Gambino,
M. Sammartino,
C. Sgarra
Abstract:
In this paper the valuation problem of a European call option in presence of both stochastic volatility and transaction costs is considered. In the limit of small transaction costs and fast mean reversion, an asymptotic expression for the option price is obtained. While the dominant term in the expansion it is shown to be the classical Black and Scholes solution, the correction terms appear at…
▽ More
In this paper the valuation problem of a European call option in presence of both stochastic volatility and transaction costs is considered. In the limit of small transaction costs and fast mean reversion, an asymptotic expression for the option price is obtained. While the dominant term in the expansion it is shown to be the classical Black and Scholes solution, the correction terms appear at $O(\varepsilon^{1/2})$ and $O(\varepsilon)$. The optimal hedging strategy is then explicitly obtained for the Scott's model.
△ Less
Submitted 19 November, 2012;
originally announced November 2012.
-
Beyond the Child-Langmuir Limit
Authors:
R. E. Caflisch,
M. S. Rosin
Abstract:
This paper describes a new solution formulation for fully nonlinear and unsteady planar flow of an electron beam in a diode. Using characteristic variables - i.e., variables that follow particle paths - the solution is expressed through an exact analytic, but implicit, formula for any choice of incoming velocity $v_0$, electric field $E_0$ and current $J_0$. For steady solutions, this approach cla…
▽ More
This paper describes a new solution formulation for fully nonlinear and unsteady planar flow of an electron beam in a diode. Using characteristic variables - i.e., variables that follow particle paths - the solution is expressed through an exact analytic, but implicit, formula for any choice of incoming velocity $v_0$, electric field $E_0$ and current $J_0$. For steady solutions, this approach clarifies the origin of the maximal current $J_max$, derived by Child and Langmuir for $v_0=0$ and by Jaffe for $v_0>0$. The implicit formulation is used to find (1) unsteady solutions having constant incoming flux $J_0>J_max$, which leads formation of a virtual cathode, and (2) time-periodic solutions whose average flux exceeds the adiabatic average of $J_max$.
△ Less
Submitted 13 October, 2011;
originally announced October 2011.
-
Direct simulation Monte Carlo schemes for Coulomb interactions in plasmas
Authors:
Giacomo Dimarco,
Russell Caflisch,
Lorenzo Pareschi
Abstract:
We consider the development of Monte Carlo schemes for molecules with Coulomb interactions. We generalize the classic algorithms of Bird and Nanbu-Babovsky for rarefied gas dynamics to the Coulomb case thanks to the approximation introduced by Bobylev and Nanbu (Theory of collision algorithms for gases and plasmas based on the Boltzmann equation and the Landau-Fokker-Planck equation, Physical Revi…
▽ More
We consider the development of Monte Carlo schemes for molecules with Coulomb interactions. We generalize the classic algorithms of Bird and Nanbu-Babovsky for rarefied gas dynamics to the Coulomb case thanks to the approximation introduced by Bobylev and Nanbu (Theory of collision algorithms for gases and plasmas based on the Boltzmann equation and the Landau-Fokker-Planck equation, Physical Review E, Vol. 61, 2000). Thus, instead of considering the original Boltzmann collision operator, the schemes are constructed through the use of an approximated Boltzmann operator. With the above choice larger time steps are possible in simulations; moreover the expensive acceptance-rejection procedure for collisions is avoided and every particle collides. Error analysis and comparisons with the original Bobylev-Nanbu (BN) scheme are performed. The numerical results show agreement with the theoretical convergence rate of the approximated Boltzmann operator and the better performance of Bird-type schemes with respect to the original scheme.
△ Less
Submitted 4 October, 2010; v1 submitted 1 October, 2010;
originally announced October 2010.
-
Anisotropic step stiffness from a kinetic model of epitaxial growth
Authors:
Dionisios Margetis,
Russel E. Caflisch
Abstract:
Starting from a detailed model for the kinetics of a step edge or island boundary, we derive a Gibbs-Thomson type formula and the associated step stiffness as a function of the step edge orientation angle, $theta$. Basic ingredients of the model are: (i) the diffusion of point defects (``adatoms'') on terraces and along step edges; (ii) the convection of kinks along step edges; and (iii) constit…
▽ More
Starting from a detailed model for the kinetics of a step edge or island boundary, we derive a Gibbs-Thomson type formula and the associated step stiffness as a function of the step edge orientation angle, $theta$. Basic ingredients of the model are: (i) the diffusion of point defects (``adatoms'') on terraces and along step edges; (ii) the convection of kinks along step edges; and (iii) constitutive laws that relate adatom fluxes, sources for kinks, and the kink velocity with densities via a mean-field approach. This model has a kinetic (nonequilibrium) steady-state solution that corresponds to epitaxial growth through step flow. The step stiffness, $\tbe(θ)$, is determined via perturbations of the kinetic steady state for small edge Peclet number, P, which is the ratio of the deposition to the diffusive flux along a step edge. In particular, $\tbe$ is found to satisfy $\tbe =O(θ^{-1})$ for $O(P^{1/3}) <θ\ll 1$, which is in agreement with independent, equilibrium-based calculations.
△ Less
Submitted 17 September, 2007;
originally announced September 2007.
-
Level Set Approach to Reversible Epitaxial Growth
Authors:
M. Petersen,
C. Ratsch,
R. E. Caflisch,
A. Zangwill
Abstract:
We generalize the level set approach to model epitaxial growth to include thermal detachment of atoms from island edges. This means that islands do not always grow and island dissociation can occur. We make no assumptions about a critical nucleus. Excellent quantitative agreement is obtained with kinetic Monte Carlo simulations for island densities and island size distributions in the submonolay…
▽ More
We generalize the level set approach to model epitaxial growth to include thermal detachment of atoms from island edges. This means that islands do not always grow and island dissociation can occur. We make no assumptions about a critical nucleus. Excellent quantitative agreement is obtained with kinetic Monte Carlo simulations for island densities and island size distributions in the submonolayer regime.
△ Less
Submitted 14 November, 2001;
originally announced November 2001.
