Skip to main content

Showing 1–18 of 18 results for author: de Bézenac, E

Searching in archive cs. Search in all archives.
.
  1. arXiv:2410.06820  [pdf, ps, other

    cs.LG

    Learning a Neural Solver for Parametric PDE to Enhance Physics-Informed Methods

    Authors: Lise Le Boudec, Emmanuel de Bezenac, Louis Serrano, Ramon Daniel Regueiro-Espino, Yuan Yin, Patrick Gallinari

    Abstract: Physics-informed deep learning often faces optimization challenges due to the complexity of solving partial differential equations (PDEs), which involve exploring large solution spaces, require numerous iterations, and can lead to unstable training. These challenges arise particularly from the ill-conditioning of the optimization problem caused by the differential terms in the loss function. To ad… ▽ More

    Submitted 2 June, 2025; v1 submitted 9 October, 2024; originally announced October 2024.

  2. arXiv:2405.19101  [pdf, other

    cs.LG

    Poseidon: Efficient Foundation Models for PDEs

    Authors: Maximilian Herde, Bogdan Raonić, Tobias Rohner, Roger Käppeli, Roberto Molinaro, Emmanuel de Bézenac, Siddhartha Mishra

    Abstract: We introduce Poseidon, a foundation model for learning the solution operators of PDEs. It is based on a multiscale operator transformer, with time-conditioned layer norms that enable continuous-in-time evaluations. A novel training strategy leveraging the semi-group property of time-dependent PDEs to allow for significant scaling-up of the training data is also proposed. Poseidon is pretrained on… ▽ More

    Submitted 5 November, 2024; v1 submitted 29 May, 2024; originally announced May 2024.

  3. arXiv:2310.05801  [pdf, other

    cs.LG

    An operator preconditioning perspective on training in physics-informed machine learning

    Authors: Tim De Ryck, Florent Bonnet, Siddhartha Mishra, Emmanuel de Bézenac

    Abstract: In this paper, we investigate the behavior of gradient descent algorithms in physics-informed machine learning methods like PINNs, which minimize residuals connected to partial differential equations (PDEs). Our key result is that the difficulty in training these models is closely related to the conditioning of a specific differential operator. This operator, in turn, is associated to the Hermitia… ▽ More

    Submitted 3 May, 2024; v1 submitted 9 October, 2023; originally announced October 2023.

  4. arXiv:2309.17357  [pdf, other

    cs.LG

    Module-wise Training of Neural Networks via the Minimizing Movement Scheme

    Authors: Skander Karkar, Ibrahim Ayed, Emmanuel de Bézenac, Patrick Gallinari

    Abstract: Greedy layer-wise or module-wise training of neural networks is compelling in constrained and on-device settings where memory is limited, as it circumvents a number of problems of end-to-end back-propagation. However, it suffers from a stagnation problem, whereby early layers overfit and deeper layers stop increasing the test accuracy after a certain depth. We propose to solve this issue by introd… ▽ More

    Submitted 5 October, 2023; v1 submitted 29 September, 2023; originally announced September 2023.

    Comments: NeurIPS 2023. arXiv admin note: text overlap with arXiv:2210.00949

  5. arXiv:2305.19913  [pdf, other

    cs.LG eess.SP

    Representation Equivalent Neural Operators: a Framework for Alias-free Operator Learning

    Authors: Francesca Bartolucci, Emmanuel de Bézenac, Bogdan Raonić, Roberto Molinaro, Siddhartha Mishra, Rima Alaifari

    Abstract: Recently, operator learning, or learning mappings between infinite-dimensional function spaces, has garnered significant attention, notably in relation to learning partial differential equations from data. Conceptually clear when outlined on paper, neural operators necessitate discretization in the transition to computer implementations. This step can compromise their integrity, often causing them… ▽ More

    Submitted 2 November, 2023; v1 submitted 31 May, 2023; originally announced May 2023.

    Comments: 28 pages

  6. arXiv:2305.19663  [pdf, other

    cs.LG math.NA

    Beyond Regular Grids: Fourier-Based Neural Operators on Arbitrary Domains

    Authors: Levi Lingsch, Mike Y. Michelis, Emmanuel de Bezenac, Sirani M. Perera, Robert K. Katzschmann, Siddhartha Mishra

    Abstract: The computational efficiency of many neural operators, widely used for learning solutions of PDEs, relies on the fast Fourier transform (FFT) for performing spectral computations. As the FFT is limited to equispaced (rectangular) grids, this limits the efficiency of such neural operators when applied to problems where the input and output functions need to be processed on general non-equispaced po… ▽ More

    Submitted 20 May, 2024; v1 submitted 31 May, 2023; originally announced May 2023.

    Comments: 20 pages, 12 figures

  7. arXiv:2305.16150  [pdf, other

    cs.LG cs.CV cs.NE stat.ML

    Unifying GANs and Score-Based Diffusion as Generative Particle Models

    Authors: Jean-Yves Franceschi, Mike Gartrell, Ludovic Dos Santos, Thibaut Issenhuth, Emmanuel de Bézenac, Mickaël Chen, Alain Rakotomamonjy

    Abstract: Particle-based deep generative models, such as gradient flows and score-based diffusion models, have recently gained traction thanks to their striking performance. Their principle of displacing particle distributions using differential equations is conventionally seen as opposed to the previously widespread generative adversarial networks (GANs), which involve training a pushforward generator netw… ▽ More

    Submitted 21 December, 2023; v1 submitted 25 May, 2023; originally announced May 2023.

    Journal ref: Thirty-seventh Conference on Neural Information Processing Systems, Neural Information Processing Systems Foundation, Dec. 2023, New Orleans, LA, USA

  8. arXiv:2302.01178  [pdf, other

    cs.LG

    Convolutional Neural Operators for robust and accurate learning of PDEs

    Authors: Bogdan Raonić, Roberto Molinaro, Tim De Ryck, Tobias Rohner, Francesca Bartolucci, Rima Alaifari, Siddhartha Mishra, Emmanuel de Bézenac

    Abstract: Although very successfully used in conventional machine learning, convolution based neural network architectures -- believed to be inconsistent in function space -- have been largely ignored in the context of learning solution operators of PDEs. Here, we present novel adaptations for convolutional neural networks to demonstrate that they are indeed able to process functions as inputs and outputs.… ▽ More

    Submitted 1 December, 2023; v1 submitted 2 February, 2023; originally announced February 2023.

  9. arXiv:2210.00949  [pdf, other

    cs.LG

    Block-wise Training of Residual Networks via the Minimizing Movement Scheme

    Authors: Skander Karkar, Ibrahim Ayed, Emmanuel de Bézenac, Patrick Gallinari

    Abstract: End-to-end backpropagation has a few shortcomings: it requires loading the entire model during training, which can be impossible in constrained settings, and suffers from three locking problems (forward locking, update locking and backward locking), which prohibit training the layers in parallel. Solving layer-wise optimization problems can address these problems and has been used in on-device tra… ▽ More

    Submitted 6 June, 2023; v1 submitted 3 October, 2022; originally announced October 2022.

    Comments: 1st International Workshop on Practical Deep Learning in the Wild at AAAI 2022

  10. arXiv:2110.15057  [pdf, other

    cs.LG cs.AI

    Mapping conditional distributions for domain adaptation under generalized target shift

    Authors: Matthieu Kirchmeyer, Alain Rakotomamonjy, Emmanuel de Bezenac, Patrick Gallinari

    Abstract: We consider the problem of unsupervised domain adaptation (UDA) between a source and a target domain under conditional and label shift a.k.a Generalized Target Shift (GeTarS). Unlike simpler UDA settings, few works have addressed this challenging problem. Recent approaches learn domain-invariant representations, yet they have practical limitations and rely on strong assumptions that may not hold i… ▽ More

    Submitted 18 March, 2022; v1 submitted 26 October, 2021; originally announced October 2021.

  11. arXiv:2106.05566  [pdf, other

    cs.LG cs.NE stat.ML

    A Neural Tangent Kernel Perspective of GANs

    Authors: Jean-Yves Franceschi, Emmanuel de Bézenac, Ibrahim Ayed, Mickaël Chen, Sylvain Lamprier, Patrick Gallinari

    Abstract: We propose a novel theoretical framework of analysis for Generative Adversarial Networks (GANs). We reveal a fundamental flaw of previous analyses which, by incorrectly modeling GANs' training scheme, are subject to ill-defined discriminator gradients. We overcome this issue which impedes a principled study of GAN training, solving it within our framework by taking into account the discriminator's… ▽ More

    Submitted 7 November, 2022; v1 submitted 10 June, 2021; originally announced June 2021.

    Journal ref: 39th International Conference on Machine Learning, International Machine Learning Society, Jul 2022, Baltimore, MD, United States. pp.6660-6704

  12. arXiv:2106.04546  [pdf, other

    cs.LG cs.AI stat.ML

    LEADS: Learning Dynamical Systems that Generalize Across Environments

    Authors: Yuan Yin, Ibrahim Ayed, Emmanuel de Bézenac, Nicolas Baskiotis, Patrick Gallinari

    Abstract: When modeling dynamical systems from real-world data samples, the distribution of data often changes according to the environment in which they are captured, and the dynamics of the system itself vary from one environment to another. Generalizing across environments thus challenges the conventional frameworks. The classical settings suggest either considering data as i.i.d. and learning a single m… ▽ More

    Submitted 14 February, 2022; v1 submitted 8 June, 2021; originally announced June 2021.

    Comments: Published at NeurIPS 2021

  13. arXiv:2010.04456  [pdf, other

    stat.ML cs.AI cs.CV cs.LG

    Augmenting Physical Models with Deep Networks for Complex Dynamics Forecasting

    Authors: Yuan Yin, Vincent Le Guen, Jérémie Dona, Emmanuel de Bézenac, Ibrahim Ayed, Nicolas Thome, Patrick Gallinari

    Abstract: Forecasting complex dynamical phenomena in settings where only partial knowledge of their dynamics is available is a prevalent problem across various scientific fields. While purely data-driven approaches are arguably insufficient in this context, standard physical modeling based approaches tend to be over-simplistic, inducing non-negligible errors. In this work, we introduce the APHYNITY framewor… ▽ More

    Submitted 10 May, 2022; v1 submitted 9 October, 2020; originally announced October 2020.

    Comments: Accepted at ICLR 2021 (Oral)

    Journal ref: J. Stat. Mech. (2021) 124012

  14. arXiv:2009.08372  [pdf, other

    stat.ML cs.LG

    A Principle of Least Action for the Training of Neural Networks

    Authors: Skander Karkar, Ibrahim Ayed, Emmanuel de Bézenac, Patrick Gallinari

    Abstract: Neural networks have been achieving high generalization performance on many tasks despite being highly over-parameterized. Since classical statistical learning theory struggles to explain this behavior, much effort has recently been focused on uncovering the mechanisms behind it, in the hope of developing a more adequate theoretical framework and having a better control over the trained models. In… ▽ More

    Submitted 15 June, 2021; v1 submitted 17 September, 2020; originally announced September 2020.

    Comments: ECML PKDD 2020

  15. arXiv:1912.12164  [pdf, other

    cs.CV cs.LG eess.IV stat.ML

    Unsupervised Adversarial Image Inpainting

    Authors: Arthur Pajot, Emmanuel de Bezenac, Patrick Gallinari

    Abstract: We consider inpainting in an unsupervised setting where there is neither access to paired nor unpaired training data. The only available information is provided by the uncomplete observations and the inpainting process statistics. In this context, an observation should give rise to several plausible reconstructions which amounts at learning a distribution over the space of reconstructed images. We… ▽ More

    Submitted 18 December, 2019; originally announced December 2019.

  16. arXiv:1906.01292  [pdf, other

    cs.LG cs.CV stat.ML

    Optimal Unsupervised Domain Translation

    Authors: Emmanuel de Bézenac, Ibrahim Ayed, Patrick Gallinari

    Abstract: Domain Translation is the problem of finding a meaningful correspondence between two domains. Since in a majority of settings paired supervision is not available, much work focuses on Unsupervised Domain Translation (UDT) where data samples from each domain are unpaired. Following the seminal work of CycleGAN for UDT, many variants and extensions of this model have been proposed. However, there is… ▽ More

    Submitted 4 June, 2019; originally announced June 2019.

  17. arXiv:1902.11136  [pdf, other

    eess.SY cs.LG math.DS physics.ao-ph

    Learning Dynamical Systems from Partial Observations

    Authors: Ibrahim Ayed, Emmanuel de Bézenac, Arthur Pajot, Julien Brajard, Patrick Gallinari

    Abstract: We consider the problem of forecasting complex, nonlinear space-time processes when observations provide only partial information of on the system's state. We propose a natural data-driven framework, where the system's dynamics are modelled by an unknown time-varying differential equation, and the evolution term is estimated from the data, using a neural network. Any future state can then be compu… ▽ More

    Submitted 26 February, 2019; originally announced February 2019.

  18. arXiv:1711.07970  [pdf, other

    cs.AI cs.LG stat.ML

    Deep Learning for Physical Processes: Incorporating Prior Scientific Knowledge

    Authors: Emmanuel de Bezenac, Arthur Pajot, Patrick Gallinari

    Abstract: We consider the use of Deep Learning methods for modeling complex phenomena like those occurring in natural physical processes. With the large amount of data gathered on these phenomena the data intensive paradigm could begin to challenge more traditional approaches elaborated over the years in fields like maths or physics. However, despite considerable successes in a variety of application domain… ▽ More

    Submitted 9 January, 2018; v1 submitted 21 November, 2017; originally announced November 2017.