Showing 1–15 of 15 results for author: Pei, F

Autonomous Behavior and Whole-Brain Dynamics Emerge in Embodied Zebrafish Agents with Model-based Intrinsic Motivation

Authors: Reece Keller, Alyn Tornell, Felix Pei, Xaq Pitkow, Leo Kozachkov, Aran Nayebi

Abstract: Autonomy is a hallmark of animal intelligence, enabling adaptive and intelligent behavior in complex environments without relying on external reward or task structure. Existing reinforcement learning approaches to exploration in sparse reward and reward-free environments, including class of methods known as intrinsic motivation, exhibit inconsistent exploration patterns and thus fail to produce ro… ▽ More Autonomy is a hallmark of animal intelligence, enabling adaptive and intelligent behavior in complex environments without relying on external reward or task structure. Existing reinforcement learning approaches to exploration in sparse reward and reward-free environments, including class of methods known as intrinsic motivation, exhibit inconsistent exploration patterns and thus fail to produce robust autonomous behaviors observed in animals. Moreover, systems neuroscience has largely overlooked the neural basis of autonomy, focusing instead on experimental paradigms where animals are motivated by external reward rather than engaging in unconstrained, naturalistic and task-independent behavior. To bridge these gaps, we introduce a novel model-based intrinsic drive explicitly designed to capture robust autonomous exploration observed in animals. Our method (3M-Progress) motivates naturalistic behavior by tracking divergence between the agent's current world model and an ethological prior. We demonstrate that artificial embodied agents trained with 3M-Progress capture the explainable variance in behavioral patterns and whole-brain neural-glial dynamics recorded from autonomously-behaving larval zebrafish, introducing the first goal-driven, population-level model of neural-glial computation. Our findings establish a computational framework connecting model-based intrinsic motivation to naturalistic behavior, providing a foundation for building artificial agents with animal-like autonomy. △ Less

Submitted 30 May, 2025; originally announced June 2025.

Comments: 17 pages, 7 figures

sbi reloaded: a toolkit for simulation-based inference workflows

Authors: Jan Boelts, Michael Deistler, Manuel Gloeckler, Álvaro Tejero-Cantero, Jan-Matthis Lueckmann, Guy Moss, Peter Steinbach, Thomas Moreau, Fabio Muratore, Julia Linhart, Conor Durkan, Julius Vetter, Benjamin Kurt Miller, Maternus Herold, Abolfazl Ziaeemehr, Matthijs Pals, Theo Gruner, Sebastian Bischoff, Nastya Krouglova, Richard Gao, Janne K. Lappalainen, Bálint Mucsányi, Felix Pei, Auguste Schulz, Zinovia Stefanidi , et al. (8 additional authors not shown)

Abstract: Scientists and engineers use simulators to model empirically observed phenomena. However, tuning the parameters of a simulator to ensure its outputs match observed data presents a significant challenge. Simulation-based inference (SBI) addresses this by enabling Bayesian inference for simulators, identifying parameters that match observed data and align with prior knowledge. Unlike traditional Bay… ▽ More Scientists and engineers use simulators to model empirically observed phenomena. However, tuning the parameters of a simulator to ensure its outputs match observed data presents a significant challenge. Simulation-based inference (SBI) addresses this by enabling Bayesian inference for simulators, identifying parameters that match observed data and align with prior knowledge. Unlike traditional Bayesian inference, SBI only needs access to simulations from the model and does not require evaluations of the likelihood-function. In addition, SBI algorithms do not require gradients through the simulator, allow for massive parallelization of simulations, and can perform inference for different observations without further simulations or training, thereby amortizing inference. Over the past years, we have developed, maintained, and extended $\texttt{sbi}$, a PyTorch-based package that implements Bayesian SBI algorithms based on neural networks. The $\texttt{sbi}$ toolkit implements a wide range of inference methods, neural network architectures, sampling methods, and diagnostic tools. In addition, it provides well-tested default settings but also offers flexibility to fully customize every step of the simulation-based inference workflow. Taken together, the $\texttt{sbi}$ toolkit enables scientists and engineers to apply state-of-the-art SBI methods to black-box simulators, opening up new possibilities for aligning simulations with empirically observed data. △ Less

Submitted 26 November, 2024; originally announced November 2024.

GS^3: Efficient Relighting with Triple Gaussian Splatting

Authors: Zoubin Bi, Yixin Zeng, Chong Zeng, Fan Pei, Xiang Feng, Kun Zhou, Hongzhi Wu

Abstract: We present a spatial and angular Gaussian based representation and a triple splatting process, for real-time, high-quality novel lighting-and-view synthesis from multi-view point-lit input images. To describe complex appearance, we employ a Lambertian plus a mixture of angular Gaussians as an effective reflectance function for each spatial Gaussian. To generate self-shadow, we splat all spatial Ga… ▽ More We present a spatial and angular Gaussian based representation and a triple splatting process, for real-time, high-quality novel lighting-and-view synthesis from multi-view point-lit input images. To describe complex appearance, we employ a Lambertian plus a mixture of angular Gaussians as an effective reflectance function for each spatial Gaussian. To generate self-shadow, we splat all spatial Gaussians towards the light source to obtain shadow values, which are further refined by a small multi-layer perceptron. To compensate for other effects like global illumination, another network is trained to compute and add a per-spatial-Gaussian RGB tuple. The effectiveness of our representation is demonstrated on 30 samples with a wide variation in geometry (from solid to fluffy) and appearance (from translucent to anisotropic), as well as using different forms of input data, including rendered images of synthetic/reconstructed objects, photographs captured with a handheld camera and a flash, or from a professional lightstage. We achieve a training time of 40-70 minutes and a rendering speed of 90 fps on a single commodity GPU. Our results compare favorably with state-of-the-art techniques in terms of quality/performance. Our code and data are publicly available at https://GSrelight.github.io/. △ Less

Submitted 15 October, 2024; originally announced October 2024.

Comments: Accepted to SIGGRAPH Asia 2024. Project page: https://gsrelight.github.io/

Journal ref: ACM SIGGRAPH Asia 2024 Conference Papers

Latent Diffusion for Neural Spiking Data

Authors: Jaivardhan Kapoor, Auguste Schulz, Julius Vetter, Felix Pei, Richard Gao, Jakob H. Macke

Abstract: Modern datasets in neuroscience enable unprecedented inquiries into the relationship between complex behaviors and the activity of many simultaneously recorded neurons. While latent variable models can successfully extract low-dimensional embeddings from such recordings, using them to generate realistic spiking data, especially in a behavior-dependent manner, still poses a challenge. Here, we pres… ▽ More Modern datasets in neuroscience enable unprecedented inquiries into the relationship between complex behaviors and the activity of many simultaneously recorded neurons. While latent variable models can successfully extract low-dimensional embeddings from such recordings, using them to generate realistic spiking data, especially in a behavior-dependent manner, still poses a challenge. Here, we present Latent Diffusion for Neural Spiking data (LDNS), a diffusion-based generative model with a low-dimensional latent space: LDNS employs an autoencoder with structured state-space (S4) layers to project discrete high-dimensional spiking data into continuous time-aligned latents. On these inferred latents, we train expressive (conditional) diffusion models, enabling us to sample neural activity with realistic single-neuron and population spiking statistics. We validate LDNS on synthetic data, accurately recovering latent structure, firing rates, and spiking statistics. Next, we demonstrate its flexibility by generating variable-length data that mimics human cortical activity during attempted speech. We show how to equip LDNS with an expressive observation model that accounts for single-neuron dynamics not mediated by the latent state, further increasing the realism of generated samples. Finally, conditional LDNS trained on motor cortical activity during diverse reaching behaviors can generate realistic spiking data given reach direction or unseen reach trajectories. In summary, LDNS simultaneously enables inference of low-dimensional latents and realistic conditional generation of neural spiking datasets, opening up further possibilities for simulating experimentally testable hypotheses. △ Less

Submitted 2 December, 2024; v1 submitted 27 June, 2024; originally announced July 2024.

Comments: 38th Conference on Neural Information Processing Systems (NeurIPS 2024)

Inferring stochastic low-rank recurrent neural networks from neural data

Authors: Matthijs Pals, A Erdem Sağtekin, Felix Pei, Manuel Gloeckler, Jakob H Macke

Abstract: A central aim in computational neuroscience is to relate the activity of large populations of neurons to an underlying dynamical system. Models of these neural dynamics should ideally be both interpretable and fit the observed data well. Low-rank recurrent neural networks (RNNs) exhibit such interpretability by having tractable dynamics. However, it is unclear how to best fit low-rank RNNs to data… ▽ More A central aim in computational neuroscience is to relate the activity of large populations of neurons to an underlying dynamical system. Models of these neural dynamics should ideally be both interpretable and fit the observed data well. Low-rank recurrent neural networks (RNNs) exhibit such interpretability by having tractable dynamics. However, it is unclear how to best fit low-rank RNNs to data consisting of noisy observations of an underlying stochastic system. Here, we propose to fit stochastic low-rank RNNs with variational sequential Monte Carlo methods. We validate our method on several datasets consisting of both continuous and spiking neural data, where we obtain lower dimensional latent dynamics than current state of the art methods. Additionally, for low-rank models with piecewise linear nonlinearities, we show how to efficiently identify all fixed points in polynomial rather than exponential cost in the number of units, making analysis of the inferred dynamics tractable for large RNNs. Our method both elucidates the dynamical systems underlying experimental recordings and provides a generative model whose trajectories match observed variability. △ Less

Submitted 26 February, 2025; v1 submitted 24 June, 2024; originally announced June 2024.

Journal ref: The Thirty-eighth Annual Conference on Neural Information Processing Systems (NeurIPS) 2024

A Practical Guide to Sample-based Statistical Distances for Evaluating Generative Models in Science

Authors: Sebastian Bischoff, Alana Darcher, Michael Deistler, Richard Gao, Franziska Gerken, Manuel Gloeckler, Lisa Haxel, Jaivardhan Kapoor, Janne K Lappalainen, Jakob H Macke, Guy Moss, Matthijs Pals, Felix Pei, Rachel Rapp, A Erdem Sağtekin, Cornelius Schröder, Auguste Schulz, Zinovia Stefanidi, Shoji Toyota, Linda Ulmer, Julius Vetter

Abstract: Generative models are invaluable in many fields of science because of their ability to capture high-dimensional and complicated distributions, such as photo-realistic images, protein structures, and connectomes. How do we evaluate the samples these models generate? This work aims to provide an accessible entry point to understanding popular sample-based statistical distances, requiring only founda… ▽ More Generative models are invaluable in many fields of science because of their ability to capture high-dimensional and complicated distributions, such as photo-realistic images, protein structures, and connectomes. How do we evaluate the samples these models generate? This work aims to provide an accessible entry point to understanding popular sample-based statistical distances, requiring only foundational knowledge in mathematics and statistics. We focus on four commonly used notions of statistical distances representing different methodologies: Using low-dimensional projections (Sliced-Wasserstein; SW), obtaining a distance using classifiers (Classifier Two-Sample Tests; C2ST), using embeddings through kernels (Maximum Mean Discrepancy; MMD), or neural networks (Fréchet Inception Distance; FID). We highlight the intuition behind each distance and explain their merits, scalability, complexity, and pitfalls. To demonstrate how these distances are used in practice, we evaluate generative models from different scientific domains, namely a model of decision-making and a model generating medical images. We showcase that distinct distances can give different results on similar data. Through this guide, we aim to help researchers to use, interpret, and evaluate statistical distances for generative models in science. △ Less

Submitted 10 October, 2024; v1 submitted 19 March, 2024; originally announced March 2024.

Journal ref: Transactions on Machine Learning Research (TMLR) 2024

Learning Photometric Feature Transform for Free-form Object Scan

Authors: Xiang Feng, Kaizhang Kang, Fan Pei, Huakeng Ding, Jinjiang You, Ping Tan, Kun Zhou, Hongzhi Wu

Abstract: We propose a novel framework to automatically learn to aggregate and transform photometric measurements from multiple unstructured views into spatially distinctive and view-invariant low-level features, which are subsequently fed to a multi-view stereo pipeline to enhance 3D reconstruction. The illumination conditions during acquisition and the feature transform are jointly trained on a large amou… ▽ More We propose a novel framework to automatically learn to aggregate and transform photometric measurements from multiple unstructured views into spatially distinctive and view-invariant low-level features, which are subsequently fed to a multi-view stereo pipeline to enhance 3D reconstruction. The illumination conditions during acquisition and the feature transform are jointly trained on a large amount of synthetic data. We further build a system to reconstruct both the geometry and anisotropic reflectance of a variety of challenging objects from hand-held scans. The effectiveness of the system is demonstrated with a lightweight prototype, consisting of a camera and an array of LEDs, as well as an off-the-shelf tablet. Our results are validated against reconstructions from a professional 3D scanner and photographs, and compare favorably with state-of-the-art techniques. △ Less

Submitted 10 December, 2024; v1 submitted 7 August, 2023; originally announced August 2023.

Braiding lateral morphotropic grain boundary in homogeneitic oxides

Authors: Shengru Chen, Qinghua Zhang, Dongke Rong, Yue Xu, Jinfeng Zhang, Fangfang Pei, He Bai, Yan-Xing Shang, Shan Lin, Qiao Jin, Haitao Hong, Can Wang, Wensheng Yan, Haizhong Guo, Tao Zhu, Lin Gu, Yu Gong, Qian Li, Lingfei Wang, Gang-Qin Liu, Kui-juan Jin, Er-Jia Guo

Abstract: Interfaces formed by correlated oxides offer a critical avenue for discovering emergent phenomena and quantum states. However, the fabrication of oxide interfaces with variable crystallographic orientations and strain states integrated along a film plane is extremely challenge by conventional layer-by-layer stacking or self-assembling. Here, we report the creation of morphotropic grain boundaries… ▽ More Interfaces formed by correlated oxides offer a critical avenue for discovering emergent phenomena and quantum states. However, the fabrication of oxide interfaces with variable crystallographic orientations and strain states integrated along a film plane is extremely challenge by conventional layer-by-layer stacking or self-assembling. Here, we report the creation of morphotropic grain boundaries (GBs) in laterally interconnected cobaltite homostructures. Single-crystalline substrates and suspended ultrathin freestanding membranes provide independent templates for coherent epitaxy and constraint on the growth orientation, resulting in seamless and atomically sharp GBs. Electronic states and magnetic behavior in hybrid structures are laterally modulated and isolated by GBs, enabling artificially engineered functionalities in the planar matrix. Our work offers a simple and scalable method for fabricating unprecedented innovative interfaces through controlled synthesis routes as well as provides a platform for exploring potential applications in neuromorphics, solid state batteries, and catalysis. △ Less

Submitted 13 July, 2022; originally announced July 2022.

Comments: 38 pages, 4 main figures

Neural Latents Benchmark '21: Evaluating latent variable models of neural population activity

Authors: Felix Pei, Joel Ye, David Zoltowski, Anqi Wu, Raeed H. Chowdhury, Hansem Sohn, Joseph E. O'Doherty, Krishna V. Shenoy, Matthew T. Kaufman, Mark Churchland, Mehrdad Jazayeri, Lee E. Miller, Jonathan Pillow, Il Memming Park, Eva L. Dyer, Chethan Pandarinath

Abstract: Advances in neural recording present increasing opportunities to study neural activity in unprecedented detail. Latent variable models (LVMs) are promising tools for analyzing this rich activity across diverse neural systems and behaviors, as LVMs do not depend on known relationships between the activity and external experimental variables. However, progress with LVMs for neuronal population activ… ▽ More Advances in neural recording present increasing opportunities to study neural activity in unprecedented detail. Latent variable models (LVMs) are promising tools for analyzing this rich activity across diverse neural systems and behaviors, as LVMs do not depend on known relationships between the activity and external experimental variables. However, progress with LVMs for neuronal population activity is currently impeded by a lack of standardization, resulting in methods being developed and compared in an ad hoc manner. To coordinate these modeling efforts, we introduce a benchmark suite for latent variable modeling of neural population activity. We curate four datasets of neural spiking activity from cognitive, sensory, and motor areas to promote models that apply to the wide variety of activity seen across these areas. We identify unsupervised evaluation as a common framework for evaluating models across datasets, and apply several baselines that demonstrate benchmark diversity. We release this benchmark through EvalAI. http://neurallatents.github.io △ Less

Submitted 17 January, 2022; v1 submitted 9 September, 2021; originally announced September 2021.

Conductance through a helical state in an InSb nanowire

Authors: Jakob Kammhuber, Maja C Cassidy, Fei Pei, Michal P Nowak, Adriaan Vuik, Diana Car, Sèbastien R Plissard, Erik P A M Bakkers, Michael Wimmer, Leo P Kouwenhoven

Abstract: The motion of an electron and its spin are generally not coupled. However in a one dimensional material with strong spin-orbit interaction (SOI) a helical state may emerge at finite magnetic fields, where electrons of opposite spin will have opposite momentum. The existence of this helical state has applications for spin filtering and Cooper pair splitter devices and is an essential ingredient for… ▽ More The motion of an electron and its spin are generally not coupled. However in a one dimensional material with strong spin-orbit interaction (SOI) a helical state may emerge at finite magnetic fields, where electrons of opposite spin will have opposite momentum. The existence of this helical state has applications for spin filtering and Cooper pair splitter devices and is an essential ingredient for realizing topologically protected quantum computing using Majorana zero modes. Here we report electrical conductance measurements of a quantum point contact (QPC) formed in an indium antimonide nanowire as a function of magnetic field. At magnetic fields exceeding 3T, the $2e^2/h$ plateau shows a reentrant conductance feature towards $1e^2/h$ which increases linearly in width with magnetic field before enveloping the $1e^2/h$ plateau. Rotating the external magnetic field either parallel or perpendicular to the spin-orbit field allows us to clearly attribute this experimental signature to SOI. We compare our observations with a model of a QPC incorporating SOI and extract a spin-orbit energy of ~6.5meV, which is significantly stronger than the SO energy obtained by other methods. △ Less

Submitted 24 January, 2017; originally announced January 2017.

Journal ref: Nature Communications 8, 478 (2017)

Conductance Quantization at zero magnetic field in InSb nanowires

Authors: Jakob Kammhuber, Maja C. Cassidy, Hao Zhang, Önder Gül, Fei Pei, Michiel W. A. de Moor, Bas Nijholt, Kenji Watanabe, Takashi Taniguchi, Diana Car, Sebastien R. Plissard, Erik P. A. M. Bakkers, Leo P. Kouwenhoven

Abstract: Ballistic electron transport is a key requirement for existence of a topological phase transition in proximitized InSb nanowires. However, measurements of quantized conductance as direct evidence of ballistic transport have so far been obscured due to the increased chance of backscattering in one dimensional nanowires. We show that by improving the nanowire-metal interface as well as the dielectri… ▽ More Ballistic electron transport is a key requirement for existence of a topological phase transition in proximitized InSb nanowires. However, measurements of quantized conductance as direct evidence of ballistic transport have so far been obscured due to the increased chance of backscattering in one dimensional nanowires. We show that by improving the nanowire-metal interface as well as the dielectric environment we can consistently achieve conductance quantization at zero magnetic field. Additionally, studying the sub-band evolution in a rotating magnetic field reveals an orbital degeneracy between the second and third sub-bands for perpendicular fields above 1T. △ Less

Submitted 11 March, 2016; originally announced March 2016.

Journal ref: Nano lett. 16(6), 3482-3486 (2016)

Large spin-orbit coupling in carbon nanotubes

Authors: G. A. Steele, F. Pei, E. A. Laird, J. M. Jol, H. B. Meerwaldt, L. P. Kouwenhoven

Abstract: It has recently been recognized that the strong spin-orbit interaction present in solids can lead to new phenomena, such as materials with non-trivial topological order. Although the atomic spin-orbit coupling in carbon is weak, the spin-orbit coupling in carbon nanotubes can be significant due to their curved surface. Previous works have reported spin-orbit couplings in reasonable agreement with… ▽ More It has recently been recognized that the strong spin-orbit interaction present in solids can lead to new phenomena, such as materials with non-trivial topological order. Although the atomic spin-orbit coupling in carbon is weak, the spin-orbit coupling in carbon nanotubes can be significant due to their curved surface. Previous works have reported spin-orbit couplings in reasonable agreement with theory, and this coupling strength has formed the basis of a large number of theoretical proposals. Here we report a spin-orbit coupling in three carbon nanotube devices that is an order of magnitude larger than measured before. We find a zero-field spin splitting of up to 3.4 meV, corresponding to a built-in effective magnetic field of 29 T aligned along the nanotube axis. While the origin of the large spin-orbit coupling is not explained by existing theories, its strength is promising for applications of the spin-orbit interaction in carbon nanotubes devices. △ Less

Submitted 11 April, 2013; originally announced April 2013.

Journal ref: Nature Communications 4, Article number: 1573, doi:10.1038/ncomms2584 (2013)

A valley-spin qubit in a carbon nanotube

Authors: Edward A. Laird, Fei Pei, Leo. P. Kouwenhoven

Abstract: Although electron spins in III-V semiconductor quantum dots have shown great promise as qubits, a major challenge is the unavoidable hyperfine decoherence in these materials. In group IV semiconductors, the dominant nuclear species are spinless, allowing for qubit coherence times that have been extended up to seconds in diamond and silicon. Carbon nanotubes are a particularly attractive host mater… ▽ More Although electron spins in III-V semiconductor quantum dots have shown great promise as qubits, a major challenge is the unavoidable hyperfine decoherence in these materials. In group IV semiconductors, the dominant nuclear species are spinless, allowing for qubit coherence times that have been extended up to seconds in diamond and silicon. Carbon nanotubes are a particularly attractive host material, because the spin-orbit interaction with the valley degree of freedom allows for electrical manipulation of the qubit. In this work, we realise such a qubit in a nanotube double quantum dot. The qubit is encoded in two valley-spin states, with coherent manipulation via electrically driven spin resonance (EDSR) mediated by a bend in the nanotube. Readout is performed by measuring the current in Pauli blockade. Arbitrary qubit rotations are demonstrated, and the coherence time is measured via Hahn echo. Although the measured decoherence time is only 65 ns in our current device, this work offers the possibility of creating a qubit for which hyperfine interaction can be virtually eliminated. △ Less

Submitted 10 October, 2012; originally announced October 2012.

Journal ref: Nature Nanotechnology 8, 565 (2013)

Valley-spin blockade and spin resonance in carbon nanotubes

Authors: Fei Pei, Edward A. Laird, Gary A. Steele, Leo P. Kouwenhoven

Abstract: Manipulation and readout of spin qubits in quantum dots made in III-V materials successfully rely on Pauli blockade that forbids transitions between spin-triplet and spin-singlet states. Quantum dots in group IV materials have the advantage of avoiding decoherence from the hyperfine interaction by purifying them with only zero-spin nuclei. Complications of group IV materials arise from the valley… ▽ More Manipulation and readout of spin qubits in quantum dots made in III-V materials successfully rely on Pauli blockade that forbids transitions between spin-triplet and spin-singlet states. Quantum dots in group IV materials have the advantage of avoiding decoherence from the hyperfine interaction by purifying them with only zero-spin nuclei. Complications of group IV materials arise from the valley degeneracies in the electronic bandstructure. These lead to complicated multiplet states even for two-electron quantum dots thereby significantly weakening the selection rules for Pauli blockade. Only recently have spin qubits been realized in silicon devices where the valley degeneracy is lifted by strain and spatial confinement. In carbon nanotubes Pauli blockade can be observed by lifting valley degeneracy through disorder. In clean nanotubes, quantum dots have to be made ultra-small to obtain a large energy difference between the relevant multiplet states. Here we report on low-disorder nanotubes and demonstrate Pauli blockade based on both valley and spin selection rules. We exploit the bandgap of the nanotube to obtain a large level spacing and thereby a robust blockade. Single-electron spin resonance is detected using the blockade. △ Less

Submitted 9 October, 2012; originally announced October 2012.

Comments: 31 pages including supplementary information

Journal ref: Nature Nanotechnology 7, 630 - 634 (2012)

Ab initio study on the electronic and magnetic properties of CaFe2As2 within a GGA + negative U approach

Authors: S. Wu, G. Wang, F. Pei, S. Y. Wang, L. Y. Chen, C. Z. Wang, K. M. Ho

Abstract: This paper has been withdrawn by the author due to the error in figure 2, we did not optimized c-value of the supercell, so the results can not give the true physics of this material. This paper has been withdrawn by the author due to the error in figure 2, we did not optimized c-value of the supercell, so the results can not give the true physics of this material. △ Less

Submitted 14 January, 2009; v1 submitted 12 January, 2009; originally announced January 2009.

Comments: This paper has been withdrawn