-
Active Learning Enhanced Surrogate Modeling of Jet Engines in JuliaSim
Authors:
Anas Abdelrehim,
Dhairya Gandhi,
Sharan Yalburgi,
Ashutosh Bharambe,
Ranjan Anantharaman,
Chris Rackauckas
Abstract:
Surrogate models are effective tools for accelerated design of complex systems. The result of a design optimization procedure using surrogate models can be used to initialize an optimization routine using the full order system. High accuracy of the surrogate model can be advantageous for fast convergence. In this work, we present an active learning approach to produce a very high accuracy surrogat…
▽ More
Surrogate models are effective tools for accelerated design of complex systems. The result of a design optimization procedure using surrogate models can be used to initialize an optimization routine using the full order system. High accuracy of the surrogate model can be advantageous for fast convergence. In this work, we present an active learning approach to produce a very high accuracy surrogate model of a turbofan jet engine, that demonstrates 0.1\% relative error for all quantities of interest. We contrast this with a surrogate model produced using a more traditional brute-force data generation approach.
△ Less
Submitted 13 January, 2025;
originally announced January 2025.
-
The Llama 3 Herd of Models
Authors:
Aaron Grattafiori,
Abhimanyu Dubey,
Abhinav Jauhri,
Abhinav Pandey,
Abhishek Kadian,
Ahmad Al-Dahle,
Aiesha Letman,
Akhil Mathur,
Alan Schelten,
Alex Vaughan,
Amy Yang,
Angela Fan,
Anirudh Goyal,
Anthony Hartshorn,
Aobo Yang,
Archi Mitra,
Archie Sravankumar,
Artem Korenev,
Arthur Hinsvark,
Arun Rao,
Aston Zhang,
Aurelien Rodriguez,
Austen Gregerson,
Ava Spataru,
Baptiste Roziere
, et al. (536 additional authors not shown)
Abstract:
Modern artificial intelligence (AI) systems are powered by foundation models. This paper presents a new set of foundation models, called Llama 3. It is a herd of language models that natively support multilinguality, coding, reasoning, and tool usage. Our largest model is a dense Transformer with 405B parameters and a context window of up to 128K tokens. This paper presents an extensive empirical…
▽ More
Modern artificial intelligence (AI) systems are powered by foundation models. This paper presents a new set of foundation models, called Llama 3. It is a herd of language models that natively support multilinguality, coding, reasoning, and tool usage. Our largest model is a dense Transformer with 405B parameters and a context window of up to 128K tokens. This paper presents an extensive empirical evaluation of Llama 3. We find that Llama 3 delivers comparable quality to leading language models such as GPT-4 on a plethora of tasks. We publicly release Llama 3, including pre-trained and post-trained versions of the 405B parameter language model and our Llama Guard 3 model for input and output safety. The paper also presents the results of experiments in which we integrate image, video, and speech capabilities into Llama 3 via a compositional approach. We observe this approach performs competitively with the state-of-the-art on image, video, and speech recognition tasks. The resulting models are not yet being broadly released as they are still under development.
△ Less
Submitted 23 November, 2024; v1 submitted 31 July, 2024;
originally announced July 2024.
-
NeuralPDE: Automating Physics-Informed Neural Networks (PINNs) with Error Approximations
Authors:
Kirill Zubov,
Zoe McCarthy,
Yingbo Ma,
Francesco Calisto,
Valerio Pagliarino,
Simone Azeglio,
Luca Bottero,
Emmanuel Luján,
Valentin Sulzer,
Ashutosh Bharambe,
Nand Vinchhi,
Kaushik Balakrishnan,
Devesh Upadhyay,
Chris Rackauckas
Abstract:
Physics-informed neural networks (PINNs) are an increasingly powerful way to solve partial differential equations, generate digital twins, and create neural surrogates of physical models. In this manuscript we detail the inner workings of NeuralPDE.jl and show how a formulation structured around numerical quadrature gives rise to new loss functions which allow for adaptivity towards bounded error…
▽ More
Physics-informed neural networks (PINNs) are an increasingly powerful way to solve partial differential equations, generate digital twins, and create neural surrogates of physical models. In this manuscript we detail the inner workings of NeuralPDE.jl and show how a formulation structured around numerical quadrature gives rise to new loss functions which allow for adaptivity towards bounded error tolerances. We describe the various ways one can use the tool, detailing mathematical techniques like using extended loss functions for parameter estimation and operator discovery, to help potential users adopt these PINN-based techniques into their workflow. We showcase how NeuralPDE uses a purely symbolic formulation so that all of the underlying training code is generated from an abstract formulation, and show how to make use of GPUs and solve systems of PDEs. Afterwards we give a detailed performance analysis which showcases the trade-off between training techniques on a large set of PDEs. We end by focusing on a complex multiphysics example, the Doyle-Fuller-Newman (DFN) Model, and showcase how this PDE can be formulated and solved with NeuralPDE. Together this manuscript is meant to be a detailed and approachable technical report to help potential users of the technique quickly get a sense of the real-world performance trade-offs and use cases of the PINN techniques.
△ Less
Submitted 19 July, 2021;
originally announced July 2021.
-
Exploring the Limits of Weakly Supervised Pretraining
Authors:
Dhruv Mahajan,
Ross Girshick,
Vignesh Ramanathan,
Kaiming He,
Manohar Paluri,
Yixuan Li,
Ashwin Bharambe,
Laurens van der Maaten
Abstract:
State-of-the-art visual perception models for a wide range of tasks rely on supervised pretraining. ImageNet classification is the de facto pretraining task for these models. Yet, ImageNet is now nearly ten years old and is by modern standards "small". Even so, relatively little is known about the behavior of pretraining with datasets that are multiple orders of magnitude larger. The reasons are o…
▽ More
State-of-the-art visual perception models for a wide range of tasks rely on supervised pretraining. ImageNet classification is the de facto pretraining task for these models. Yet, ImageNet is now nearly ten years old and is by modern standards "small". Even so, relatively little is known about the behavior of pretraining with datasets that are multiple orders of magnitude larger. The reasons are obvious: such datasets are difficult to collect and annotate. In this paper, we present a unique study of transfer learning with large convolutional networks trained to predict hashtags on billions of social media images. Our experiments demonstrate that training for large-scale hashtag prediction leads to excellent results. We show improvements on several image classification and object detection tasks, and report the highest ImageNet-1k single-crop, top-1 accuracy to date: 85.4% (97.6% top-5). We also perform extensive experiments that provide novel empirical data on the relationship between large-scale pretraining and transfer learning performance.
△ Less
Submitted 2 May, 2018;
originally announced May 2018.
