-
Learning to reset in target search problems
Authors:
Gorka Muñoz-Gil,
Hans J. Briegel,
Michele Caraglio
Abstract:
Target search problems are central to a wide range of fields, from biological foraging to the optimization algorithms. Recently, the ability to reset the search has been shown to significantly improve the searcher's efficiency. However, the optimal resetting strategy depends on the specific properties of the search problem and can often be challenging to determine. In this work, we propose a reinf…
▽ More
Target search problems are central to a wide range of fields, from biological foraging to the optimization algorithms. Recently, the ability to reset the search has been shown to significantly improve the searcher's efficiency. However, the optimal resetting strategy depends on the specific properties of the search problem and can often be challenging to determine. In this work, we propose a reinforcement learning (RL)-based framework to train agents capable of optimizing their search efficiency in environments by learning how to reset. First, we validate the approach in a well-established benchmark: the Brownian search with resetting. There, RL agents consistently recover strategies closely resembling the sharp resetting distribution, known to be optimal in this scenario. We then extend the framework by allowing agents to control not only when to reset, but also their spatial dynamics through turning actions. In this more complex setting, the agents discover strategies that adapt both resetting and turning to the properties of the environment, outperforming the proposed benchmarks. These results demonstrate how reinforcement learning can serve both as an optimization tool and a mechanism for uncovering new, interpretable strategies in stochastic search processes with resetting.
△ Less
Submitted 14 March, 2025;
originally announced March 2025.
-
Quantitative evaluation of methods to analyze motion changes in single-particle experiments
Authors:
Gorka Muñoz-Gil,
Harshith Bachimanchi,
Jesús Pineda,
Benjamin Midtvedt,
Maciej Lewenstein,
Ralf Metzler,
Diego Krapf,
Giovanni Volpe,
Carlo Manzo
Abstract:
The analysis of live-cell single-molecule imaging experiments can reveal valuable information about the heterogeneity of transport processes and interactions between cell components. These characteristics are seen as motion changes in the particle trajectories. Despite the existence of multiple approaches to carry out this type of analysis, no objective assessment of these methods has been perform…
▽ More
The analysis of live-cell single-molecule imaging experiments can reveal valuable information about the heterogeneity of transport processes and interactions between cell components. These characteristics are seen as motion changes in the particle trajectories. Despite the existence of multiple approaches to carry out this type of analysis, no objective assessment of these methods has been performed so far. Here, we have designed a competition to characterize and rank the performance of these methods when analyzing the dynamic behavior of single molecules. To run this competition, we have implemented a software library to simulate realistic data corresponding to widespread diffusion and interaction models, both in the form of trajectories and videos obtained in typical experimental conditions. The competition will constitute the first assessment of these methods, provide insights into the current limits of the field, foster the development of new approaches, and guide researchers to identify optimal tools for analyzing their experiments.
△ Less
Submitted 19 January, 2024; v1 submitted 29 November, 2023;
originally announced November 2023.
-
Learning how to find targets in the micro-world: The case of intermittent active Brownian particles
Authors:
Michele Caraglio,
Harpreet Kaur,
Lukas J. Fiderer,
Andrea López-Incera,
Hans J. Briegel,
Thomas Franosch,
Gorka Muñoz-Gil
Abstract:
Finding the best strategy to minimize the time needed to find a given target is a crucial task both in nature and in reaching decisive technological advances. By considering learning agents able to switch their dynamics between standard and active Brownian motion, here we focus on developing effective target-search behavioral policies for microswimmers navigating a homogeneous environment and sear…
▽ More
Finding the best strategy to minimize the time needed to find a given target is a crucial task both in nature and in reaching decisive technological advances. By considering learning agents able to switch their dynamics between standard and active Brownian motion, here we focus on developing effective target-search behavioral policies for microswimmers navigating a homogeneous environment and searching for targets of unknown position. We exploit Projective Simulation, a reinforcement learning algorithm, to acquire an efficient stochastic policy represented by the probability of switching the phase, i.e. the navigation mode, in response to the type and the duration of the current phase. Our findings reveal that the target-search efficiency increases with the particle's self-propulsion during the active phase and that, while the optimal duration of the passive case decreases monotonically with the activity, the optimal duration of the active phase displays a non-monotonic behavior.
△ Less
Submitted 28 November, 2023;
originally announced November 2023.
-
Learning minimal representations of stochastic processes with variational autoencoders
Authors:
Gabriel Fernández-Fernández,
Carlo Manzo,
Maciej Lewenstein,
Alexandre Dauphin,
Gorka Muñoz-Gil
Abstract:
Stochastic processes have found numerous applications in science, as they are broadly used to model a variety of natural phenomena. Due to their intrinsic randomness and uncertainty, they are however difficult to characterize. Here, we introduce an unsupervised machine learning approach to determine the minimal set of parameters required to effectively describe the dynamics of a stochastic process…
▽ More
Stochastic processes have found numerous applications in science, as they are broadly used to model a variety of natural phenomena. Due to their intrinsic randomness and uncertainty, they are however difficult to characterize. Here, we introduce an unsupervised machine learning approach to determine the minimal set of parameters required to effectively describe the dynamics of a stochastic process. Our method builds upon an extended $β$-variational autoencoder architecture. By means of simulated datasets corresponding to paradigmatic diffusion models, we showcase its effectiveness in extracting the minimal relevant parameters that accurately describe these dynamics. Furthermore, the method enables the generation of new trajectories that faithfully replicate the expected stochastic behavior. Overall, our approach enables for the autonomous discovery of unknown parameters describing stochastic processes, hence enhancing our comprehension of complex phenomena across various fields.
△ Less
Submitted 4 August, 2023; v1 submitted 21 July, 2023;
originally announced July 2023.
-
Optimal foraging strategies can be learned
Authors:
Gorka Muñoz-Gil,
Andrea López-Incera,
Lukas J. Fiderer,
Hans J. Briegel
Abstract:
The foraging behavior of animals is a paradigm of target search in nature. Understanding which foraging strategies are optimal and how animals learn them are central challenges in modeling animal foraging. While the question of optimality has wide-ranging implications across fields such as economy, physics, and ecology, the question of learnability is a topic of ongoing debate in evolutionary biol…
▽ More
The foraging behavior of animals is a paradigm of target search in nature. Understanding which foraging strategies are optimal and how animals learn them are central challenges in modeling animal foraging. While the question of optimality has wide-ranging implications across fields such as economy, physics, and ecology, the question of learnability is a topic of ongoing debate in evolutionary biology. Recognizing the interconnected nature of these challenges, this work addresses them simultaneously by exploring optimal foraging strategies through a reinforcement learning framework. To this end, we model foragers as learning agents. We first prove theoretically that maximizing rewards in our reinforcement learning model is equivalent to optimizing foraging efficiency. We then show with numerical experiments that, in the paradigmatic model of non-destructive search, our agents learn foraging strategies which outperform the efficiency of some of the best known strategies such as Lévy walks. These findings highlight the potential of reinforcement learning as a versatile framework not only for optimizing search strategies but also to model the learning process, thus shedding light on the role of learning in natural optimization processes.
△ Less
Submitted 3 August, 2023; v1 submitted 10 March, 2023;
originally announced March 2023.
-
Inferring pointwise diffusion properties of single trajectories with deep learning
Authors:
Borja Requena,
Sergi Masó,
Joan Bertran,
Maciej Lewenstein,
Carlo Manzo,
Gorka Muñoz-Gil
Abstract:
In order to characterize the mechanisms governing the diffusion of particles in biological scenarios, it is essential to accurately determine their diffusive properties. To do so, we propose a machine learning method to characterize diffusion processes with time-dependent properties at the experimental time resolution. Our approach operates at the single-trajectory level predicting the properties…
▽ More
In order to characterize the mechanisms governing the diffusion of particles in biological scenarios, it is essential to accurately determine their diffusive properties. To do so, we propose a machine learning method to characterize diffusion processes with time-dependent properties at the experimental time resolution. Our approach operates at the single-trajectory level predicting the properties of interest, such as the diffusion coefficient or the anomalous diffusion exponent, at every time step of the trajectory. In this way, changes in the diffusive properties occurring along the trajectory emerge naturally in the prediction, and thus allow the characterization without any prior knowledge or assumption about the system. We first benchmark the method on synthetic trajectories simulated under several conditions. We show that our approach can successfully characterize both abrupt and continuous changes in the diffusion coefficient or the anomalous diffusion exponent. Finally, we leverage the method to analyze experiments of single-molecule diffusion of two membrane proteins in living cells: the pathogen-recognition receptor DC-SIGN and the integrin $\alpha5\beta1$. The analysis allows us to characterize physical parameters and diffusive states with unprecedented accuracy, shedding new light on the underlying mechanisms.
△ Less
Submitted 1 February, 2023;
originally announced February 2023.
-
Preface: Characterisation of Physical Processes from Anomalous Diffusion Data
Authors:
Carlo Manzo,
Gorka Muñoz-Gil,
Giovanni Volpe,
Miguel Angel Garcia-March,
Maciej Lewenstein,
Ralf Metzler
Abstract:
Preface to the special issue "Characterisation of Physical Processes from Anomalous Diffusion Data" associated with the Anomalous Diffusion Challenge ( https://andi-challenge.org ) and published in Journal of Physics A: Mathematical and Theoretical. The list of articles included in the special issue can be accessed at https://iopscience.iop.org/journal/1751-8121/page/Characterisation-of-Physical-P…
▽ More
Preface to the special issue "Characterisation of Physical Processes from Anomalous Diffusion Data" associated with the Anomalous Diffusion Challenge ( https://andi-challenge.org ) and published in Journal of Physics A: Mathematical and Theoretical. The list of articles included in the special issue can be accessed at https://iopscience.iop.org/journal/1751-8121/page/Characterisation-of-Physical-Processes-from-Anomalous-Diffusion-Data .
△ Less
Submitted 7 January, 2023; v1 submitted 2 January, 2023;
originally announced January 2023.
-
Unsupervised learning of anomalous diffusion data
Authors:
Gorka Muñoz-Gil,
Guillem Guigó i Corominas,
Maciej Lewenstein
Abstract:
The characterization of diffusion processes is a keystone in our understanding of a variety of physical phenomena. Many of these deviate from Brownian motion, giving rise to anomalous diffusion. Various theoretical models exists nowadays to describe such processes, but their application to experimental setups is often challenging, due to the stochastic nature of the phenomena and the difficulty to…
▽ More
The characterization of diffusion processes is a keystone in our understanding of a variety of physical phenomena. Many of these deviate from Brownian motion, giving rise to anomalous diffusion. Various theoretical models exists nowadays to describe such processes, but their application to experimental setups is often challenging, due to the stochastic nature of the phenomena and the difficulty to harness reliable data. The latter often consists on short and noisy trajectories, which are hard to characterize with usual statistical approaches. In recent years, we have witnessed an impressive effort to bridge theory and experiments by means of supervised machine learning techniques, with astonishing results. In this work, we explore the use of unsupervised methods in anomalous diffusion data. We show that the main diffusion characteristics can be learnt without the need of any labelling of the data. We use such method to discriminate between anomalous diffusion models and extract their physical parameters. Moreover, we explore the feasibility of finding novel types of diffusion, in this case represented by compositions of existing diffusion models. At last, we showcase the use of the method in experimental data and demonstrate its advantages for cases where supervised learning is not applicable.
△ Less
Submitted 7 August, 2021;
originally announced August 2021.
-
Objective comparison of methods to decode anomalous diffusion
Authors:
Gorka Muñoz-Gil,
Giovanni Volpe,
Miguel Angel Garcia-March,
Erez Aghion,
Aykut Argun,
Chang Beom Hong,
Tom Bland,
Stefano Bo,
J. Alberto Conejero,
Nicolás Firbas,
Òscar Garibo i Orts,
Alessia Gentili,
Zihan Huang,
Jae-Hyung Jeon,
Hélène Kabbech,
Yeongjin Kim,
Patrycja Kowalek,
Diego Krapf,
Hanna Loch-Olszewska,
Michael A. Lomholt,
Jean-Baptiste Masson,
Philipp G. Meyer,
Seongyu Park,
Borja Requena,
Ihor Smal
, et al. (9 additional authors not shown)
Abstract:
Deviations from Brownian motion leading to anomalous diffusion are ubiquitously found in transport dynamics, playing a crucial role in phenomena from quantum physics to life sciences. The detection and characterization of anomalous diffusion from the measurement of an individual trajectory are challenging tasks, which traditionally rely on calculating the mean squared displacement of the trajector…
▽ More
Deviations from Brownian motion leading to anomalous diffusion are ubiquitously found in transport dynamics, playing a crucial role in phenomena from quantum physics to life sciences. The detection and characterization of anomalous diffusion from the measurement of an individual trajectory are challenging tasks, which traditionally rely on calculating the mean squared displacement of the trajectory. However, this approach breaks down for cases of important practical interest, e.g., short or noisy trajectories, ensembles of heterogeneous trajectories, or non-ergodic processes. Recently, several new approaches have been proposed, mostly building on the ongoing machine-learning revolution. Aiming to perform an objective comparison of methods, we gathered the community and organized an open competition, the Anomalous Diffusion challenge (AnDi). Participating teams independently applied their own algorithms to a commonly-defined dataset including diverse conditions. Although no single method performed best across all scenarios, the results revealed clear differences between the various approaches, providing practical advice for users and a benchmark for developers.
△ Less
Submitted 14 May, 2021;
originally announced May 2021.
-
AnDi: The Anomalous Diffusion Challenge
Authors:
Gorka Muñoz-Gil,
Giovanni Volpe,
Miguel Angel Garcia-March,
Ralf Metzler,
Maciej Lewenstein,
Carlo Manzo
Abstract:
The deviation from pure Brownian motion generally referred to as anomalous diffusion has received large attention in the scientific literature to describe many physical scenarios. Several methods, based on classical statistics and machine learning approaches, have been developed to characterize anomalous diffusion from experimental data, which are usually acquired as particle trajectories. With th…
▽ More
The deviation from pure Brownian motion generally referred to as anomalous diffusion has received large attention in the scientific literature to describe many physical scenarios. Several methods, based on classical statistics and machine learning approaches, have been developed to characterize anomalous diffusion from experimental data, which are usually acquired as particle trajectories. With the aim to assess and compare the available methods to characterize anomalous diffusion, we have organized the Anomalous Diffusion (AnDi) Challenge (\url{http://www.andi-challenge.org/}). Specifically, the AnDi Challenge will address three different aspects of anomalous diffusion characterization, namely: (i) Inference of the anomalous diffusion exponent. (ii) Identification of the underlying diffusion model. (iii) Segmentation of trajectories. Each problem includes sub-tasks for different number of dimensions (1D, 2D and 3D). In order to compare the various methods, we have developed a dedicated open-source framework for the simulation of the anomalous diffusion trajectories that are used for the training and test datasets. The challenge was launched on March 1, 2020, and consists of three phases. Currently, the participation to the first phase is open. Submissions will be automatically evaluated and the performance of the top-scoring methods will be thoroughly analyzed and compared in an upcoming article.
△ Less
Submitted 26 March, 2020;
originally announced March 2020.
-
Machine learning method for single trajectory characterization
Authors:
Gorka Muñoz-Gil,
Miguel Angel Garcia-March,
Carlo Manzo,
José D. Martín-Guerrero,
Maciej Lewenstein
Abstract:
In order to study transport in complex environments, it is extremely important to determine the physical mechanism underlying diffusion, and precisely characterize its nature and parameters. Often, this task is strongly impacted by data consisting of trajectories with short length and limited localization precision. In this paper, we propose a machine learning method based on a random forest archi…
▽ More
In order to study transport in complex environments, it is extremely important to determine the physical mechanism underlying diffusion, and precisely characterize its nature and parameters. Often, this task is strongly impacted by data consisting of trajectories with short length and limited localization precision. In this paper, we propose a machine learning method based on a random forest architecture, which is able to associate even very short trajectories to the underlying diffusion mechanism with a high accuracy. In addition, the method is able to classify the motion according to normal or anomalous diffusion, and determine its anomalous exponent with a small error. The method provides highly accurate outputs even when working with very short trajectories and in the presence of experimental noise. We further demonstrate the application of transfer learning to experimental and simulated data not included in the training/testing dataset. This allows for a full, high-accuracy characterization of experimental trajectories without the need of any prior information.
△ Less
Submitted 7 January, 2020; v1 submitted 7 March, 2019;
originally announced March 2019.
-
Diffusion through a network of compartments separated by partially-transmitting boundaries
Authors:
Gorka Muñoz-Gil,
Miguel Angel García-March,
Carlo Manzo,
Alessio Celi,
Maciej Lewenstein
Abstract:
We study the random walk of a particle in a compartmentalized environment, as realized in biological samples or solid state compounds. Each compartment is characterized by its length $L$ and the boundaries transmittance $T$. We identify two relevant spatio-temporal scales that provide alternative descriptions of the dynamics: i) the microscale, in which the particle position is monitored at consta…
▽ More
We study the random walk of a particle in a compartmentalized environment, as realized in biological samples or solid state compounds. Each compartment is characterized by its length $L$ and the boundaries transmittance $T$. We identify two relevant spatio-temporal scales that provide alternative descriptions of the dynamics: i) the microscale, in which the particle position is monitored at constant time intervals; and ii) the mesoscale, in which it is monitored only when the particle crosses a boundary between compartments. Both descriptions provide --by construction-- the same long time behavior. The analytical description obtained at the proposed mesoscale allows for a complete characterization of the complex movement at the microscale, thus representing a fruitful approach for this kind of systems. We show that the presence of disorder in the transmittances is a necessary condition to induce anomalous diffusion, whereas the spatial heterogeneity reduces the degree of subdiffusion and, in some cases, can even compensate for the disorder induced by the stochastic transmittance.
△ Less
Submitted 4 October, 2018;
originally announced October 2018.
-
Transient subdiffusion from an Ising environment
Authors:
G. Muñoz-Gil,
C. Charalambous,
M. A. García-March,
M. F. García-Parajo,
C. Manzo,
M. Lewenstein,
A. Celi
Abstract:
We introduce a model, in which a particle performs a continuous time random walk (CTRW) coupled to an environment with Ising dynamics. The particle shows locally varying diffusivity determined by the geometrical properties of the underlying Ising environment, that is, the diffusivity depends on the size of the connected area of spins pointing in the same direction. The model shows anomalous diffus…
▽ More
We introduce a model, in which a particle performs a continuous time random walk (CTRW) coupled to an environment with Ising dynamics. The particle shows locally varying diffusivity determined by the geometrical properties of the underlying Ising environment, that is, the diffusivity depends on the size of the connected area of spins pointing in the same direction. The model shows anomalous diffusion when the Ising environment is at critical temperature. We show that any finite scale introduced by a temperature different from the critical one, or a finite size of the environment, cause subdiffusion only during a transient time. The characteristic time, at which the system returns to normal diffusion after the subdiffusive plateau depends on the limiting scale and on how close the temperature is to criticality. The system also displays apparent ergodicity breaking at intermediate time, while ergodicity breaking at longer time occurs only under the idealized infinite environment at the critical temperature.
△ Less
Submitted 8 January, 2018; v1 submitted 3 August, 2017;
originally announced August 2017.
-
Nonergodic subdiffusion from transient interactions with heterogeneous partners
Authors:
C. Charalambous,
G. Muñoz-Gil,
A. Celi,
M. F. Garcia-Parajo,
M. Lewenstein,
C. Manzo,
M. A. García-March
Abstract:
Spatiotemporal disorder has been recently associated to the occurrence of anomalous nonergodic diffusion of molecular components in biological systems, but the underlying microscopic mechanism is still unclear. We introduce a model in which a particle performs continuous Brownian motion with changes of diffusion coefficients induced by transient molecular interactions with diffusive binding partne…
▽ More
Spatiotemporal disorder has been recently associated to the occurrence of anomalous nonergodic diffusion of molecular components in biological systems, but the underlying microscopic mechanism is still unclear. We introduce a model in which a particle performs continuous Brownian motion with changes of diffusion coefficients induced by transient molecular interactions with diffusive binding partners. In spite of the exponential distribution of waiting times, the model shows subdiffusion and nonergodicity similar to the heavy-tailed continuous time random walk. The dependence of these properties on the density of binding partners is analyzed and discussed. Our work provide an experimentally-testable microscopic model to investigate the nature of nonergodicity in disordered media.
△ Less
Submitted 3 March, 2017; v1 submitted 1 July, 2016;
originally announced July 2016.
