-
ChemGymRL: An Interactive Framework for Reinforcement Learning for Digital Chemistry
Authors:
Chris Beeler,
Sriram Ganapathi Subramanian,
Kyle Sprague,
Nouha Chatti,
Colin Bellinger,
Mitchell Shahen,
Nicholas Paquin,
Mark Baula,
Amanuel Dawit,
Zihan Yang,
Xinkai Li,
Mark Crowley,
Isaac Tamblyn
Abstract:
This paper provides a simulated laboratory for making use of Reinforcement Learning (RL) for chemical discovery. Since RL is fairly data intensive, training agents `on-the-fly' by taking actions in the real world is infeasible and possibly dangerous. Moreover, chemical processing and discovery involves challenges which are not commonly found in RL benchmarks and therefore offer a rich space to wor…
▽ More
This paper provides a simulated laboratory for making use of Reinforcement Learning (RL) for chemical discovery. Since RL is fairly data intensive, training agents `on-the-fly' by taking actions in the real world is infeasible and possibly dangerous. Moreover, chemical processing and discovery involves challenges which are not commonly found in RL benchmarks and therefore offer a rich space to work in. We introduce a set of highly customizable and open-source RL environments, ChemGymRL, based on the standard Open AI Gym template. ChemGymRL supports a series of interconnected virtual chemical benches where RL agents can operate and train. The paper introduces and details each of these benches using well-known chemical reactions as illustrative examples, and trains a set of standard RL algorithms in each of these benches. Finally, discussion and comparison of the performances of several standard RL methods are provided in addition to a list of directions for future work as a vision for the further development and usage of ChemGymRL.
△ Less
Submitted 23 May, 2023;
originally announced May 2023.
-
Dynamic programming with incomplete information to overcome navigational uncertainty in a nautical environment
Authors:
Chris Beeler,
Xinkai Li,
Colin Bellinger,
Mark Crowley,
Maia Fraser,
Isaac Tamblyn
Abstract:
Using a novel toy nautical navigation environment, we show that dynamic programming can be used when only incomplete information about a partially observed Markov decision process (POMDP) is known. By incorporating uncertainty into our model, we show that navigation policies can be constructed that maintain safety, outperforming the baseline performance of traditional dynamic programming for Marko…
▽ More
Using a novel toy nautical navigation environment, we show that dynamic programming can be used when only incomplete information about a partially observed Markov decision process (POMDP) is known. By incorporating uncertainty into our model, we show that navigation policies can be constructed that maintain safety, outperforming the baseline performance of traditional dynamic programming for Markov decision processes (MDPs). Adding in controlled sensing methods, we show that these policies can also lower measurement costs at the same time.
△ Less
Submitted 19 July, 2022; v1 submitted 29 December, 2021;
originally announced December 2021.
-
Optimizing thermodynamic trajectories using evolutionary and gradient-based reinforcement learning
Authors:
Chris Beeler,
Uladzimir Yahorau,
Rory Coles,
Kyle Mills,
Stephen Whitelam,
Isaac Tamblyn
Abstract:
Using a model heat engine, we show that neural network-based reinforcement learning can identify thermodynamic trajectories of maximal efficiency. We consider both gradient and gradient-free reinforcement learning. We use an evolutionary learning algorithm to evolve a population of neural networks, subject to a directive to maximize the efficiency of a trajectory composed of a set of elementary th…
▽ More
Using a model heat engine, we show that neural network-based reinforcement learning can identify thermodynamic trajectories of maximal efficiency. We consider both gradient and gradient-free reinforcement learning. We use an evolutionary learning algorithm to evolve a population of neural networks, subject to a directive to maximize the efficiency of a trajectory composed of a set of elementary thermodynamic processes; the resulting networks learn to carry out the maximally-efficient Carnot, Stirling, or Otto cycles. When given an additional irreversible process, this evolutionary scheme learns a previously unknown thermodynamic cycle. Gradient-based reinforcement learning is able to learn the Stirling cycle, whereas an evolutionary approach achieves the optimal Carnot cycle. Our results show how the reinforcement learning strategies developed for game playing can be applied to solve physical problems conditioned upon path-extensive order parameters.
△ Less
Submitted 22 November, 2021; v1 submitted 20 March, 2019;
originally announced March 2019.
-
Extensive deep neural networks for transferring small scale learning to large scale systems
Authors:
Kyle Mills,
Kevin Ryczko,
Iryna Luchak,
Adam Domurad,
Chris Beeler,
Isaac Tamblyn
Abstract:
We present a physically-motivated topology of a deep neural network that can efficiently infer extensive parameters (such as energy, entropy, or number of particles) of arbitrarily large systems, doing so with O(N) scaling. We use a form of domain decomposition for training and inference, where each sub-domain (tile) is comprised of a non-overlapping focus region surrounded by an overlapping conte…
▽ More
We present a physically-motivated topology of a deep neural network that can efficiently infer extensive parameters (such as energy, entropy, or number of particles) of arbitrarily large systems, doing so with O(N) scaling. We use a form of domain decomposition for training and inference, where each sub-domain (tile) is comprised of a non-overlapping focus region surrounded by an overlapping context region. The size of these regions is motivated by the physical interaction length scales of the problem. We demonstrate the application of EDNNs to three physical systems: the Ising model and two hexagonal/graphene-like datasets. In the latter, an EDNN was able to make total energy predictions of a 60 atoms system, with comparable accuracy to density functional theory (DFT), in 57 milliseconds. Additionally EDNNs are well suited for massively parallel evaluation, as no communication is necessary during neural network evaluation. We demonstrate that EDNNs can be used to make an energy prediction of a two-dimensional 35.2 million atom system, over 1 square micrometer of material, at an accuracy comparable to DFT, in under 25 minutes. Such a system exists on a length scale visible with optical microscopy and larger than some living organisms.
△ Less
Submitted 11 April, 2019; v1 submitted 17 August, 2017;
originally announced August 2017.
-
Gauge matters: Observing the vortex-nucleation transition in a Bose condensate
Authors:
L. J. LeBlanc,
K. Jiménez-García,
R. A. Williams,
M. C. Beeler,
W. D. Phillips,
I. B. Spielman
Abstract:
The order parameter of a quantum-coherent many-body system can include a phase degree of freedom, which, in the presence of an electromagnetic field, depends on the choice of gauge. Because of the relationship between the phase gradient and the velocity, time-of-flight measurements reveal this gradient. Here, we make such measurements using initially trapped Bose-Einstein condensates (BECs) subjec…
▽ More
The order parameter of a quantum-coherent many-body system can include a phase degree of freedom, which, in the presence of an electromagnetic field, depends on the choice of gauge. Because of the relationship between the phase gradient and the velocity, time-of-flight measurements reveal this gradient. Here, we make such measurements using initially trapped Bose-Einstein condensates (BECs) subject to an artificial magnetic field. Vortices are nucleated in the BEC for artificial field strengths above a critical value, which represents a structural phase transition. By comparing to superfluid-hydrodynamic and Gross-Pitaevskii calculations, we confirmed that the transition from the vortex-free state gives rise to a shear in the released BEC's spatial distribution, representing a macroscopic method to measure this transition, distinct from direct measurements of vortex entry. Shear is also affected by an artificial electric field accompanying the artificial magnetic field turn-off, which depends on the details of the physical mechanism creating the artificial fields, and implies a natural choice of gauge. Measurements of this kind offer opportunities for studying phase in less-well-understood quantum gas systems.
△ Less
Submitted 26 February, 2015;
originally announced February 2015.
-
Tunable Spin-Orbit Coupling via Strong Driving in Ultracold Atom Systems
Authors:
K. Jiménez-García,
L. J. LeBlanc,
R. A. Williams,
M. C. Beeler,
C. Qu,
M. Gong,
C. Zhang,
I. B. Spielman
Abstract:
Spin-orbit coupling (SOC) is an essential ingredient in topological materials, conventional and quantum-gas based alike.~Engineered spin-orbit coupling in ultracold atom systems --unique in their experimental control and measurement opportunities-- provides a major opportunity to investigate and understand topological phenomena.~Here we experimentally demonstrate and theoretically analyze a techni…
▽ More
Spin-orbit coupling (SOC) is an essential ingredient in topological materials, conventional and quantum-gas based alike.~Engineered spin-orbit coupling in ultracold atom systems --unique in their experimental control and measurement opportunities-- provides a major opportunity to investigate and understand topological phenomena.~Here we experimentally demonstrate and theoretically analyze a technique for controlling SOC in a two component Bose-Einstein condensate using amplitude-modulated Raman coupling.
△ Less
Submitted 12 December, 2014;
originally announced December 2014.
-
The spin Hall effect in a quantum gas
Authors:
M. C. Beeler,
R. A. Williams,
K. Jiménez-García,
L. J. LeBlanc,
A. R. Perry,
I. B. Spielman
Abstract:
Electronic properties like current flow are generally independent of the electron's spin angular momentum, an internal degree of freedom present in quantum particles. The spin Hall effects (SHEs), first proposed 40 years ago, are an unusual class of phenomena where flowing particles experience orthogonally directed spin-dependent Lorentz-like forces, analogous to the conventional Lorentz force for…
▽ More
Electronic properties like current flow are generally independent of the electron's spin angular momentum, an internal degree of freedom present in quantum particles. The spin Hall effects (SHEs), first proposed 40 years ago, are an unusual class of phenomena where flowing particles experience orthogonally directed spin-dependent Lorentz-like forces, analogous to the conventional Lorentz force for the Hall effect, but opposite in sign for two spin states. Such spin Hall effects have been observed for electrons flowing in spin-orbit coupled materials such as GaAs or InGaAs and for laser light traversing dielectric junctions. Here we observe the spin Hall effect in a quantum-degenerate Bose gas, and use the resulting spin-dependent Lorentz forces to realize a cold-atom spin transistor. By engineering a spatially inhomogeneous spin-orbit coupling field for our quantum gas, we explicitly introduce and measure the requisite spin-dependent Lorentz forces, in excellent agreement with our calculations. This atomtronic circuit element behaves as a new type of velocity-insensitive adiabatic spin-selector, with potential application in devices such as magnetic or inertial sensors. In addition, such techniques --- for both creating and measuring the SHE --- are clear prerequisites for engineering topological insulators and detecting their associated quantized spin Hall effects in quantum gases. As implemented, our system realized a laser-actuated analog to the Datta-Das spin transistor.
△ Less
Submitted 22 February, 2014; v1 submitted 15 June, 2013;
originally announced June 2013.
-
A Raman-induced Feshbach resonance in an effectively single-component Fermi gas
Authors:
R. A. Williams,
M. C. Beeler,
L. J. LeBlanc,
K. Jimenez-Garcia,
I. B. Spielman
Abstract:
Ultracold gases of interacting spin-orbit coupled fermions are predicted to display exotic phenomena such as topological superfluidity and its associated Majorana fermions. Here, we experimentally demonstrate a route to strongly-interacting single-component atomic Fermi gases by combining an s-wave Feshbach resonance (giving strong interactions) and spin-orbit coupling (creating an effective p-wav…
▽ More
Ultracold gases of interacting spin-orbit coupled fermions are predicted to display exotic phenomena such as topological superfluidity and its associated Majorana fermions. Here, we experimentally demonstrate a route to strongly-interacting single-component atomic Fermi gases by combining an s-wave Feshbach resonance (giving strong interactions) and spin-orbit coupling (creating an effective p-wave channel). We identify the Feshbach resonance by its associated atomic loss feature and show that, in agreement with our single-channel scattering model, this feature is preserved and shifted as a function of the spin-orbit coupling parameters.
△ Less
Submitted 8 June, 2013;
originally announced June 2013.
-
Direct observation of zitterbewegung in a Bose-Einstein condensate
Authors:
L. J. LeBlanc,
M. C. Beeler,
K. Jimenez-Garcia,
A. R. Perry,
S. Sugawa,
R. A. Williams,
I. B. Spielman
Abstract:
Zitterbewegung, a force-free trembling motion first predicted for relativistic fermions like electrons, was an unexpected consequence of the Dirac equation's unification of quantum mechanics and special relativity. Though the oscillatory motion's large frequency and small amplitude have precluded its measurement with electrons, zitterbewegung is observable via quantum simulation. We engineered an…
▽ More
Zitterbewegung, a force-free trembling motion first predicted for relativistic fermions like electrons, was an unexpected consequence of the Dirac equation's unification of quantum mechanics and special relativity. Though the oscillatory motion's large frequency and small amplitude have precluded its measurement with electrons, zitterbewegung is observable via quantum simulation. We engineered an environment for 87Rb Bose-Einstein condensates where the constituent atoms behaved like relativistic particles subject to the one-dimensional Dirac equation. With direct imaging, we observed the sub-micrometer trembling motion of these clouds, demonstrating the utility of neutral ultracold quantum gases for simulating Dirac particles.
△ Less
Submitted 3 July, 2013; v1 submitted 4 March, 2013;
originally announced March 2013.
-
The Peierls substitution in an engineered lattice potential
Authors:
K. Jiménez-García,
L. J. LeBlanc,
R. A. Williams,
M. C. Beeler,
A. R. Perry,
I. B. Spielman
Abstract:
Artificial gauge fields open new possibilities to realize quantum many-body systems with ultracold atoms, by engineering Hamiltonians usually associated with electronic systems. In the presence of a periodic potential, artificial gauge fields may bring ultracold atoms closer to the quantum Hall regime. Here, we describe a one-dimensional lattice derived purely from effective Zeeman-shifts resultin…
▽ More
Artificial gauge fields open new possibilities to realize quantum many-body systems with ultracold atoms, by engineering Hamiltonians usually associated with electronic systems. In the presence of a periodic potential, artificial gauge fields may bring ultracold atoms closer to the quantum Hall regime. Here, we describe a one-dimensional lattice derived purely from effective Zeeman-shifts resulting from a combination of Raman coupling and radiofrequency magnetic fields. In this lattice, the tunneling matrix element is generally complex. We control both the amplitude and the phase of this tunneling parameter, experimentally realizing the Peierls substitution for ultracold neutral atoms.
△ Less
Submitted 31 January, 2012;
originally announced January 2012.
-
Observation of a superfluid Hall effect
Authors:
L. J. LeBlanc,
K. Jimenez-Garcia,
R. A. Williams,
M. C. Beeler,
A. R. Perry,
W. D. Phillips,
I. B Spielman
Abstract:
Measurement techniques based upon the Hall effect are invaluable tools in condensed matter physics. When an electric current flows perpendicular to a magnetic field, a Hall voltage develops in the direction transverse to both the current and the field. In semiconductors, this behaviour is routinely used to measure the density and charge of the current carriers (electrons in conduction bands or hol…
▽ More
Measurement techniques based upon the Hall effect are invaluable tools in condensed matter physics. When an electric current flows perpendicular to a magnetic field, a Hall voltage develops in the direction transverse to both the current and the field. In semiconductors, this behaviour is routinely used to measure the density and charge of the current carriers (electrons in conduction bands or holes in valence bands) -- internal properties of the system that are not accessible from measurements of the conventional resistance. For strongly interacting electron systems, whose behaviour can be very different from the free electron gas, the Hall effect's sensitivity to internal properties makes it a powerful tool; indeed, the quantum Hall effects are named after the tool by which they are most distinctly measured instead of the physics from which the phenomena originate. Here we report the first observation of a Hall effect in an ultracold gas of neutral atoms, revealed by measuring a Bose-Einstein condensate's transport properties perpendicular to a synthetic magnetic field. Our observations in this vortex-free superfluid are in good agreement with hydrodynamic predictions, demonstrating that the system's global irrotationality influences this superfluid Hall signal.
△ Less
Submitted 27 January, 2012;
originally announced January 2012.
-
Synthetic partial waves in ultracold atomic collisions
Authors:
R. A. Williams,
L. J. LeBlanc,
K. Jimenez-Garcia,
M. C. Beeler,
A. R. Perry,
W. D. Phillips,
I. B. Spielman
Abstract:
Interactions between particles can be strongly altered by their environment. We demonstrate a technique for modifying interactions between ultracold atoms by dressing the bare atomic states with light, creating an effective interaction of vastly increased range that scatters states of finite relative angular momentum at collision energies where only s-wave scattering would normally be expected. We…
▽ More
Interactions between particles can be strongly altered by their environment. We demonstrate a technique for modifying interactions between ultracold atoms by dressing the bare atomic states with light, creating an effective interaction of vastly increased range that scatters states of finite relative angular momentum at collision energies where only s-wave scattering would normally be expected. We collided two optically dressed neutral atomic Bose-Einstein condensates with equal, and opposite, momenta and observed that the usual s-wave distribution of scattered atoms was altered by the appearance of d- and g-wave contributions. This technique is expected to enable quantum simulation of exotic systems, including those predicted to support Majorana fermions.
△ Less
Submitted 20 January, 2012;
originally announced January 2012.
-
Disorder-Driven Loss of Phase Coherence in a Quasi-2D Cold Atom System
Authors:
M. C. Beeler,
M. E. W. Reed,
T. Hong,
S. L. Rolston
Abstract:
We study the order parameter of a quasi-2D gas of ultracold atoms trapped in an optical potential in the presence of controllable disorder. Our results show that disorder drives phase fluctuations without significantly affecting the amplitude of the quasi-condensate order parameter. This is evidence that disorder can drive phase fluctuations in 2D systems, relevant to the phase-fluctuation mechani…
▽ More
We study the order parameter of a quasi-2D gas of ultracold atoms trapped in an optical potential in the presence of controllable disorder. Our results show that disorder drives phase fluctuations without significantly affecting the amplitude of the quasi-condensate order parameter. This is evidence that disorder can drive phase fluctuations in 2D systems, relevant to the phase-fluctuation mechanism for the superconductor-to-insulator phase transition (SIT) in disordered 2D superconductors.
△ Less
Submitted 22 March, 2012; v1 submitted 5 November, 2011;
originally announced November 2011.
