Showing 1–7 of 7 results for author: Earls, C

Lines of Thought in Large Language Models

Authors: Raphaël Sarfati, Toni J. B. Liu, Nicolas Boullé, Christopher J. Earls

Abstract: Large Language Models achieve next-token prediction by transporting a vectorized piece of text (prompt) across an accompanying embedding space under the action of successive transformer layers. The resulting high-dimensional trajectories realize different contextualization, or 'thinking', steps, and fully determine the output probability distribution. We aim to characterize the statistical propert… ▽ More Large Language Models achieve next-token prediction by transporting a vectorized piece of text (prompt) across an accompanying embedding space under the action of successive transformer layers. The resulting high-dimensional trajectories realize different contextualization, or 'thinking', steps, and fully determine the output probability distribution. We aim to characterize the statistical properties of ensembles of these 'lines of thought.' We observe that independent trajectories cluster along a low-dimensional, non-Euclidean manifold, and that their path can be well approximated by a stochastic equation with few parameters extracted from data. We find it remarkable that the vast complexity of such large models can be reduced to a much simpler form, and we reflect on implications. △ Less

Submitted 13 February, 2025; v1 submitted 2 October, 2024; originally announced October 2024.

A new engineering theory describing oblique free surface impact by flexible plates

Authors: Wensi Wu, Christopher Earls

Abstract: Consideration of slamming loads within the structural design of planning hulls is of critical importance in ensuring adequate structural performance in order to avoid potential catastrophic consequences. However, because of the intricacy in the interplay between complex fluid flows and nonlinear structural deformations that accompany the phenomenology of slamming, a general engineering theory in s… ▽ More Consideration of slamming loads within the structural design of planning hulls is of critical importance in ensuring adequate structural performance in order to avoid potential catastrophic consequences. However, because of the intricacy in the interplay between complex fluid flows and nonlinear structural deformations that accompany the phenomenology of slamming, a general engineering theory in slamming has yet to be uncovered, and so design relies on specialized theories. In this paper, we propose one such theory for a design case that has, until now, eluded a proper description. In pursuit of this theory, we employ a specialized implicit, partitioned fluid-structural interaction (FSI) simulation approach, in order to study the underlying physical mechanisms accompanying the oblique impact of a flexible plate during water entry. In the present work, we first present validation results from flexible plate water entry experiments, to confirm the veracity of the developed FSI solver. Subsequent to validation, we carry out a series of numerical analyses, in an effort to characterize the regimes in impact force and plate out-of-plane deformations, as a function of impact velocities and plate flexural rigidity. Finally, we use our FSI solver, as a kind of "microscope", to study the mechanistic evolution of fluid flows and elastic plate deformations that occur during slamming. Based on these observations, we propose a novel, but simple engineering theory for flexible plates obliquely impacting the water free surface (e.g. high speed porpoising water craft reentry). △ Less

Submitted 2 November, 2021; v1 submitted 14 March, 2021; originally announced March 2021.

A Principled Approach to Design Using High Fidelity Fluid-Structure Interaction Simulations

Authors: Wensi Wu, Christophe Bonneville, Christopher J. Earls

Abstract: A high fidelity fluid-structure interaction simulation may require many days to run, on hundreds of cores. This poses a serious burden, both in terms of time and economic considerations, when repetitions of such simulations may be required (e.g. for the purpose of design optimization). In this paper we present strategies based on (constrained) Bayesian optimization (BO) to alleviate this burden. B… ▽ More A high fidelity fluid-structure interaction simulation may require many days to run, on hundreds of cores. This poses a serious burden, both in terms of time and economic considerations, when repetitions of such simulations may be required (e.g. for the purpose of design optimization). In this paper we present strategies based on (constrained) Bayesian optimization (BO) to alleviate this burden. BO is a numerical optimization technique based on Gaussian processes (GP) that is able to efficiently (with minimal calls to the expensive FSI models) converge towards some globally optimal design, as gauged using a black box objective function. In this study we present a principled design evolution that moves from FSI model verification, through a series of Bridge Simulations (bringing the verification case incrementally closer to the application), in order that we may identify material properties for an underwater, unmanned, autonomous vehicle (UUAV) sail plane. We are able to achieve fast convergence towards an optimal design, using a small number of FSI simulations (a dozen at most), even when selecting over several design parameters, and while respecting optimization constraints. △ Less

Submitted 21 August, 2020; originally announced August 2020.

Deep Learning for Classifying and Characterizing Atmospheric Ducting within the Maritime Setting

Authors: Hilarie Sit, Christopher J. Earls

Abstract: Real-time characterization of refractivity within the marine atmospheric boundary layer can provide valuable information that can potentially be used to mitigate the effects of atmospheric ducting on radar performance. Many duct characterization models are successful at predicting parameters from a specific refractivity profile associated with a given type of duct; however, the ability to classify… ▽ More Real-time characterization of refractivity within the marine atmospheric boundary layer can provide valuable information that can potentially be used to mitigate the effects of atmospheric ducting on radar performance. Many duct characterization models are successful at predicting parameters from a specific refractivity profile associated with a given type of duct; however, the ability to classify, and then subsequently characterize, various duct types is an important step towards a more comprehensive prediction model. We introduce a two-step approach using deep learning to differentiate sparsely sampled propagation factor measurements collected under evaporation ducting conditions with those collected under surface-based duct conditions in order to subsequently estimate the appropriate refractivity parameters based on that differentiation. We show that this approach is not only accurate, but also efficient; thus providing a suitable method for real-time applications. △ Less

Submitted 13 May, 2020; originally announced May 2020.

Comments: 13 pages, 3 figures

Gaussian Process Regression for Estimating EM Ducting Within the Marine Atmospheric Boundary Layer

Authors: Hilarie Sit, Christopher J. Earls

Abstract: We show that Gaussian process regression (GPR) can be used to infer the electromagnetic (EM) duct height within the marine atmospheric boundary layer (MABL) from sparsely sampled propagation factors within the context of bistatic radars. We use GPR to calculate the posterior predictive distribution on the labels (i.e. duct height) from both noise-free and noise-contaminated array of propagation fa… ▽ More We show that Gaussian process regression (GPR) can be used to infer the electromagnetic (EM) duct height within the marine atmospheric boundary layer (MABL) from sparsely sampled propagation factors within the context of bistatic radars. We use GPR to calculate the posterior predictive distribution on the labels (i.e. duct height) from both noise-free and noise-contaminated array of propagation factors. For duct height inference from noise-contaminated propagation factors, we compare a naive approach, utilizing one random sample from the input distribution (i.e. disregarding the input noise), with an inverse-variance weighted approach, utilizing a few random samples to estimate the true predictive distribution. The resulting posterior predictive distributions from these two approaches are compared to a "ground truth" distribution, which is approximated using a large number of Monte-Carlo samples. The ability of GPR to yield accurate and fast duct height predictions using a few training examples indicates the suitability of the proposed method for real-time applications. △ Less

Submitted 14 December, 2019; v1 submitted 25 May, 2019; originally announced May 2019.

Comments: 15 pages, 6 figures

Characterizing Evaporation Ducts Within the Marine Atmospheric Boundary Layer Using Artificial Neural Networks

Authors: Hilarie Sit, Christopher J. Earls

Abstract: We apply a multilayer perceptron machine learning (ML) regression approach to infer electromagnetic (EM) duct heights within the marine atmospheric boundary layer (MABL) using sparsely sampled EM propagation data obtained within a bistatic context. This paper explains the rationale behind the selection of the ML network architecture, along with other model hyperparameters, in an effort to demystif… ▽ More We apply a multilayer perceptron machine learning (ML) regression approach to infer electromagnetic (EM) duct heights within the marine atmospheric boundary layer (MABL) using sparsely sampled EM propagation data obtained within a bistatic context. This paper explains the rationale behind the selection of the ML network architecture, along with other model hyperparameters, in an effort to demystify the process of arriving at a useful ML model. The resulting speed of our ML predictions of EM duct heights, using sparse data measurements within MABL, indicates the suitability of the proposed method for real-time applications. △ Less

Submitted 26 August, 2019; v1 submitted 7 March, 2019; originally announced March 2019.

Comments: 13 pages, 7 figures

A subspace pursuit method to infer refractivity in the marine atmospheric boundary layer

Authors: Marc Aurèle Gilles, Christopher Earls, David Bindel

Abstract: Inferring electromagnetic propagation characteristics within the marine atmospheric boundary layer (MABL) from data in real time is crucial for modern maritime navigation and communications. The propagation of electromagnetic waves is well modeled by a partial differential equation (PDE): a Helmholtz equation. A natural way to solve the MABL characterization inverse problem is to minimize what is… ▽ More Inferring electromagnetic propagation characteristics within the marine atmospheric boundary layer (MABL) from data in real time is crucial for modern maritime navigation and communications. The propagation of electromagnetic waves is well modeled by a partial differential equation (PDE): a Helmholtz equation. A natural way to solve the MABL characterization inverse problem is to minimize what is observed and what is predicted by the PDE. However, this optimization is difficult because it has many local minima. We propose an alternative solution that relies on the properties of the PDE but does not involve solving the full forward model. Ducted environments result in an EM field which can be decomposed into a few propagating, trapped modes. These modes are a subset of the solutions to a Sturm-Liouville eigenvalue problem. We design a new objective function that measures the distance from the observations to a subspace spanned by these eigenvectors. The resulting optimization problem is much easier than the one that arises in the standard approach, and we show how to solve the associated nonlinear eigenvalue problem efficiently, leading to a real-time method. △ Less

Submitted 18 January, 2019; originally announced January 2019.