-
Utilizing Novelty-based Evolution Strategies to Train Transformers in Reinforcement Learning
Authors:
Matyáš Lorenc
Abstract:
In this paper, we experiment with novelty-based variants of OpenAI-ES, the NS-ES and NSR-ES algorithms, and evaluate their effectiveness in training complex, transformer-based architectures designed for the problem of reinforcement learning such as Decision Transformers. We also test if we can accelerate the novelty-based training of these larger models by seeding the training by a pretrained mode…
▽ More
In this paper, we experiment with novelty-based variants of OpenAI-ES, the NS-ES and NSR-ES algorithms, and evaluate their effectiveness in training complex, transformer-based architectures designed for the problem of reinforcement learning such as Decision Transformers. We also test if we can accelerate the novelty-based training of these larger models by seeding the training by a pretrained models. By this, we build on our previous work, where we tested the ability of evolution strategies - specifically the aforementioned OpenAI-ES - to train the Decision Transformer architecture. The results were mixed. NS-ES showed progress, but it would clearly need many more iterations for it to yield interesting results. NSR-ES, on the other hand, proved quite capable of being straightforwardly used on larger models, since its performance appears as similar between the feed-forward model and Decision Transformer, as it was for the OpenAI-ES in our previous work.
△ Less
Submitted 10 February, 2025;
originally announced February 2025.
-
Utilizing Evolution Strategies to Train Transformers in Reinforcement Learning
Authors:
Matyáš Lorenc
Abstract:
We explore a capability of evolution strategies to train an agent with its policy based on a transformer architecture in a reinforcement learning setting. We performed experiments using OpenAI's highly parallelizable evolution strategy to train Decision Transformer in Humanoid locomotion environment and in the environment of Atari games, testing the ability of this black-box optimization technique…
▽ More
We explore a capability of evolution strategies to train an agent with its policy based on a transformer architecture in a reinforcement learning setting. We performed experiments using OpenAI's highly parallelizable evolution strategy to train Decision Transformer in Humanoid locomotion environment and in the environment of Atari games, testing the ability of this black-box optimization technique to train even such relatively large and complicated models (compared to those previously tested in the literature). We also proposed a method to aid the training by first pretraining the model before using the OpenAI-ES to train it further, and tested its effectiveness. The examined evolution strategy proved to be, in general, capable of achieving strong results and managed to obtain high-performing agents. Therefore, the pretraining was shown to be unnecessary; yet still, it helped us observe and formulate several further insights.
△ Less
Submitted 23 January, 2025;
originally announced January 2025.
-
Quantification of Ultrafast Nonlinear Photothermal and Photoacoustic Effects in Molecular Thin Films via Time-Domain Brillouin Scattering
Authors:
Valentin Cherruault,
Franck Camerel,
Julien Morin,
Amédée Triadon,
Nicolas Godin,
Olivier Mongin,
Jean-François Bergamini,
Antoine Vacher,
Mark G. Humphrey,
Maciej Lorenc,
Frederic Paul,
Thomas Pezeril
Abstract:
Improving the efficiency of photothermal (PT) therapies and photoacoustic (PA) imaging at the microscopic scale hinges on developing multiphoton-absorbing photothermal molecules or contrast agents that operate in the near-infrared (NIR) range. These advanced agents or molecules will enable excitation with NIR lasers, in an improved transparency range for biological tissues, while enabling minute,…
▽ More
Improving the efficiency of photothermal (PT) therapies and photoacoustic (PA) imaging at the microscopic scale hinges on developing multiphoton-absorbing photothermal molecules or contrast agents that operate in the near-infrared (NIR) range. These advanced agents or molecules will enable excitation with NIR lasers, in an improved transparency range for biological tissues, while enabling minute, highly localized spatial control of the excitation area. However, progress in this field requires innovative experimental techniques to characterize photothermal and photoacoustic effects under multiphoton excitation. In this article, we showcase a study of a model organometallic molecular compound excited via two-photon absorption (2PA) using femtosecond laser pulses. Based on a time-domain Brillouin scattering technique, well adapted for investigating ultrafast nonlinear optical absorption processes in ultrathin films on substrates, we determine the effective nonlinear absorption coefficients of the compound directly linked to PT/PA. Our findings provide a practical approach for exploring and optimizing nonlinear PT/PA absorbers and contrast agents.
△ Less
Submitted 2 June, 2025; v1 submitted 22 January, 2025;
originally announced January 2025.
-
Nonlinear optical absorption in nanoscale films revealed through ultrafast acoustics
Authors:
Ievgeniia Chaban,
Radoslaw Deska,
Gael Privault,
Elzbieta Trzop,
Maciej Lorenc,
Steven E. Kooi,
Keith A. Nelson,
Marek Samoc,
Katarzyna Matczyszyn,
Thomas Pezeril
Abstract:
Herein we describe a novel spinning pump-probe photoacoustic technique developed to study nonlinear absorption in thin films. As a test case, an organic polycrystalline thin film of quinacridone, a well-known pigment, with a thickness in the tens of nanometers range, is excited by a femtosecond laser pulse which generates a time-domain Brillouin scattering signal. This signal is directly related t…
▽ More
Herein we describe a novel spinning pump-probe photoacoustic technique developed to study nonlinear absorption in thin films. As a test case, an organic polycrystalline thin film of quinacridone, a well-known pigment, with a thickness in the tens of nanometers range, is excited by a femtosecond laser pulse which generates a time-domain Brillouin scattering signal. This signal is directly related to the strain wave launched from the film into the substrate and can be used to quantitatively extract the nonlinear optical absorption properties of the film itself. Quinacridone exhibits both quadratic and cubic laser fluence dependence regimes which we show to correspond to two- and three-photon absorption processes. This technique can be broadly applied to materials that are difficult or impossible to characterize with conventional transmittance-based measurements including materials at the nanoscale, prone to laser damage, with very weak nonlinear properties, opaque or highly scattering.
△ Less
Submitted 17 May, 2022; v1 submitted 26 November, 2021;
originally announced November 2021.
-
Strain Wave Pathway to Semiconductor-to-Metal Transition revealed by time resolved X-ray powder diffraction
Authors:
C. Mariette,
M. Lorenc,
H. Cailleau,
E. Collet,
L. Guérin,
A. Volte,
E. Trzop,
R. Bertoni,
X. Dong,
B. Lépine,
O Hernandez,
E. Janod,
L. Cario,
V. Ta Phuoc,
S. Ohkoshi,
H. Tokoro,
L. Patthey,
A. Babic,
I. Usov,
D. Ozerov,
L. Sala,
S. Ebner,
P. Böhler,
A Keller,
A. Oggenfuss
, et al. (20 additional authors not shown)
Abstract:
Thanks to the remarkable developments of ultrafast science, one of today's challenges is to modify material state by controlling with a light pulse the coherent motions that connect two different phases. Here we show how strain waves, launched by electronic and structural precursor phenomena, determine a macroscopic transformation pathway for the semiconducting-to-metal transition with large volum…
▽ More
Thanks to the remarkable developments of ultrafast science, one of today's challenges is to modify material state by controlling with a light pulse the coherent motions that connect two different phases. Here we show how strain waves, launched by electronic and structural precursor phenomena, determine a macroscopic transformation pathway for the semiconducting-to-metal transition with large volume change in bistable Ti$_3$O$_5$ nanocrystals. Femtosecond powder X-ray diffraction allowed us to quantify the structural deformations associated with the photoinduced phase transition on relevant time scales. We monitored the early intra-cell distortions around absorbing metal dimers, but also long range crystalline deformations dynamically governed by acoustic waves launched at the laser-exposed Ti$_3$O$_5$ surface. We rationalize these observations with a simplified elastic model, demonstrating that a macroscopic transformation occurs concomitantly with the propagating acoustic wavefront on the picosecond timescale, several decades earlier than the subsequent thermal processes governed by heat diffusion.
△ Less
Submitted 20 February, 2020; v1 submitted 19 February, 2020;
originally announced February 2020.
-
Ultrafast non-thermal laser excitation of gigahertz longitudinal and shear acoustic waves in spin-crossover molecular crystals [$Fe(PM-AzA)_{2}$$(NCS)_{2}$]
Authors:
T. Parpiiev,
M. Servol,
M. Lorenc,
I. Chaban,
E. Collet,
H. Cailleau,
P. Ruello,
G. Chastanet,
T. Pezeril
Abstract:
We report GHz longitudinal as well as shear acoustic phonons photoexcitation and photodetection using femtosecond laser pulses in a spin-crossover molecular crystal. From our experimental observation of time domain Brillouin scattering triggered by the photoexcitation of acoustic waves across the low-spin (LS) to high-spin (HS) thermal crossover, we reveal a link between molecular spin state and p…
▽ More
We report GHz longitudinal as well as shear acoustic phonons photoexcitation and photodetection using femtosecond laser pulses in a spin-crossover molecular crystal. From our experimental observation of time domain Brillouin scattering triggered by the photoexcitation of acoustic waves across the low-spin (LS) to high-spin (HS) thermal crossover, we reveal a link between molecular spin state and photoexcitation of coherent GHz acoustic phonons. In particular, we experimentally evidence an unconventional non-thermal pathway for the laser excitation of GHz phonons. We also provide experimental insight on the optical and mechanical parameters evolution across the LS/HS spin crossover temperature T$_{1/2}$.
△ Less
Submitted 17 October, 2017; v1 submitted 12 April, 2016;
originally announced April 2016.
-
Ultrafast Spectral Weight Transfer in RBaCo2O6-delta (R=Sm, Gd, and Tb): A Role of Electronic Correlation in Photoinduced Phase Transition
Authors:
Y. Okimoto,
T. Miyata,
M. S. Endo,
M. Kurashima,
K. Onda,
T. Ishikawa,
S. Koshihara,
M. Lorenc,
E. Collet,
H. Cailleau,
T. Arima
Abstract:
We performed femtosecond reflection spectroscopy on a series of perovskite-type cobalt oxide $R$BaCo$_2$O$_{6-δ}$ ($R$=Sm, Gd, and Tb) crystals, in which the electronic transfer was controlled by $R$.
The transient reflectivity and the optical conductivity ($σ^{\rm PI}(ω)$) obtained by Kramers-Kronig analysis showed an ultrafast change within a time resolution ($\approx 150$ fs) at room temperat…
▽ More
We performed femtosecond reflection spectroscopy on a series of perovskite-type cobalt oxide $R$BaCo$_2$O$_{6-δ}$ ($R$=Sm, Gd, and Tb) crystals, in which the electronic transfer was controlled by $R$.
The transient reflectivity and the optical conductivity ($σ^{\rm PI}(ω)$) obtained by Kramers-Kronig analysis showed an ultrafast change within a time resolution ($\approx 150$ fs) at room temperature and the appearance of signals of a hidden state different from the high temperature metallic state. The transferred spectral weight in $σ^{\rm PI}(ω)$ upon photoexcitation sensitively depended on the $R$-species, indicating an important role of electronic correlation in the photoexcited state.
△ Less
Submitted 7 March, 2011;
originally announced March 2011.
