Showing 1–15 of 15 results for author: Ditto, W L

When less is more: evolving large neural networks from small ones

Authors: Anil Radhakrishnan, John F. Lindner, Scott T. Miller, Sudeshna Sinha, William L. Ditto

Abstract: In contrast to conventional artificial neural networks, which are large and structurally static, we study feed-forward neural networks that are small and dynamic, whose nodes can be added (or subtracted) during training. A single neuronal weight in the network controls the network's size, while the weight itself is optimized by the same gradient-descent algorithm that optimizes the network's other… ▽ More In contrast to conventional artificial neural networks, which are large and structurally static, we study feed-forward neural networks that are small and dynamic, whose nodes can be added (or subtracted) during training. A single neuronal weight in the network controls the network's size, while the weight itself is optimized by the same gradient-descent algorithm that optimizes the network's other weights and biases, but with a size-dependent objective or loss function. We train and evaluate such Nimble Neural Networks on nonlinear regression and classification tasks where they outperform the corresponding static networks. Growing networks to minimal, appropriate, or optimal sizes while training elucidates network dynamics and contrasts with pruning large networks after training but before deployment. △ Less

Submitted 29 January, 2025; originally announced January 2025.

Comments: 8 pages, 7 figures

Solving two-dimensional quantum eigenvalue problems using physics-informed machine learning

Authors: Elliott G. Holliday, John F. Lindner, William L. Ditto

Abstract: A particle confined to an impassable box is a paradigmatic and exactly solvable one-dimensional quantum system modeled by an infinite square well potential. Here we explore some of its infinitely many generalizations to two dimensions, including particles confined to rectangle, elliptic, triangle, and cardioid-shaped boxes, using physics-informed neural networks. In particular, we generalize an un… ▽ More A particle confined to an impassable box is a paradigmatic and exactly solvable one-dimensional quantum system modeled by an infinite square well potential. Here we explore some of its infinitely many generalizations to two dimensions, including particles confined to rectangle, elliptic, triangle, and cardioid-shaped boxes, using physics-informed neural networks. In particular, we generalize an unsupervised learning algorithm to find the particles' eigenvalues and eigenfunctions. During training, the neural network adjusts its weights and biases, one of which is the energy eigenvalue, so its output approximately solves the Schrödinger equation with normalized and mutually orthogonal eigenfunctions. The same procedure solves the Helmholtz equation for the harmonics and vibration modes of waves on drumheads or transverse magnetic modes of electromagnetic cavities. Related applications include dynamical billiards, quantum chaos, and Laplacian spectra. △ Less

Submitted 2 February, 2023; originally announced February 2023.

Neuronal diversity can improve machine learning for physics and beyond

Authors: Anshul Choudhary, Anil Radhakrishnan, John F. Lindner, Sudeshna Sinha, William L. Ditto

Abstract: Diversity conveys advantages in nature, yet homogeneous neurons typically comprise the layers of artificial neural networks. Here we construct neural networks from neurons that learn their own activation functions, quickly diversify, and subsequently outperform their homogeneous counterparts on image classification and nonlinear regression tasks. Sub-networks instantiate the neurons, which meta-le… ▽ More Diversity conveys advantages in nature, yet homogeneous neurons typically comprise the layers of artificial neural networks. Here we construct neural networks from neurons that learn their own activation functions, quickly diversify, and subsequently outperform their homogeneous counterparts on image classification and nonlinear regression tasks. Sub-networks instantiate the neurons, which meta-learn especially efficient sets of nonlinear responses. Examples include conventional neural networks classifying digits and forecasting a van der Pol oscillator and physics-informed Hamiltonian neural networks learning Hénon-Heiles stellar orbits and the swing of a video recorded pendulum clock. Such \textit{learned diversity} provides examples of dynamical systems selecting diversity over uniformity and elucidates the role of diversity in natural and artificial systems. △ Less

Submitted 30 August, 2023; v1 submitted 8 April, 2022; originally announced April 2022.

Comments: 13 pages, 9 figures

Forecasting Hamiltonian dynamics without canonical coordinates

Authors: Anshul Choudhary, John F. Lindner, Elliott G. Holliday, Scott T. Miller, Sudeshna Sinha, William L. Ditto

Abstract: Conventional neural networks are universal function approximators, but because they are unaware of underlying symmetries or physical laws, they may need impractically many training data to approximate nonlinear dynamics. Recently introduced Hamiltonian neural networks can efficiently learn and forecast dynamical systems that conserve energy, but they require special inputs called canonical coordin… ▽ More Conventional neural networks are universal function approximators, but because they are unaware of underlying symmetries or physical laws, they may need impractically many training data to approximate nonlinear dynamics. Recently introduced Hamiltonian neural networks can efficiently learn and forecast dynamical systems that conserve energy, but they require special inputs called canonical coordinates, which may be hard to infer from data. Here we significantly expand the scope of such networks by demonstrating a simple way to train them with any set of generalised coordinates, including easily observable ones. △ Less

Submitted 28 October, 2020; originally announced October 2020.

Comments: 7 pages, 4 figures

Mastering high-dimensional dynamics with Hamiltonian neural networks

Authors: Scott T. Miller, John F. Lindner, Anshul Choudhary, Sudeshna Sinha, William L. Ditto

Abstract: We detail how incorporating physics into neural network design can significantly improve the learning and forecasting of dynamical systems, even nonlinear systems of many dimensions. A map building perspective elucidates the superiority of Hamiltonian neural networks over conventional neural networks. The results clarify the critical relation between data, dimension, and neural network learning pe… ▽ More We detail how incorporating physics into neural network design can significantly improve the learning and forecasting of dynamical systems, even nonlinear systems of many dimensions. A map building perspective elucidates the superiority of Hamiltonian neural networks over conventional neural networks. The results clarify the critical relation between data, dimension, and neural network learning performance. △ Less

Submitted 28 July, 2020; originally announced August 2020.

Comments: 7 pages, 9 figures

Physics enhanced neural networks predict order and chaos

Authors: Anshul Choudhary, John F. Lindner, Elliott G. Holliday, Scott T. Miller, Sudeshna Sinha, William L. Ditto

Abstract: Conventional artificial neural networks are powerful tools in science and industry, but they can fail when applied to nonlinear systems where order and chaos coexist. We use neural networks that incorporate the structures and symmetries of Hamiltonian dynamics to predict phase space trajectories even as nonlinear systems transition from order to chaos. We demonstrate Hamiltonian neural networks on… ▽ More Conventional artificial neural networks are powerful tools in science and industry, but they can fail when applied to nonlinear systems where order and chaos coexist. We use neural networks that incorporate the structures and symmetries of Hamiltonian dynamics to predict phase space trajectories even as nonlinear systems transition from order to chaos. We demonstrate Hamiltonian neural networks on the canonical Henon-Heiles system, which models diverse dynamics from astrophysics to chemistry. The power of the technique and the ubiquity of chaos suggest widespread utility. △ Less

Submitted 26 November, 2019; originally announced December 2019.

Comments: 5 pages and 5 figures

Journal ref: Phys. Rev. E 101, 062207 (2020)

Chaotic Attractor Hopping yields Logic Operations

Authors: K. Murali, Sudeshna Sinha, Vivek Kohar, Behnam Kia, William L. Ditto

Abstract: Certain nonlinear systems can switch between dynamical attractors occupying different regions of phase space, under variation of parameters or initial states. In this work we exploit this feature to obtain reliable logic operations. With logic output 0 or 1 mapped to dynamical attractors bounded in distinct regions of phase space, and logic inputs encoded by a very small bias parameter, we explici… ▽ More Certain nonlinear systems can switch between dynamical attractors occupying different regions of phase space, under variation of parameters or initial states. In this work we exploit this feature to obtain reliable logic operations. With logic output 0 or 1 mapped to dynamical attractors bounded in distinct regions of phase space, and logic inputs encoded by a very small bias parameter, we explicitly demonstrate that the system hops consistently in response to an external input stream, operating effectively as a reliable logic gate. This system offers the advantage that very low-amplitude inputs yield highly amplified outputs. Additionally, different dynamical variables in the system yield complementary logic operations in parallel. Further, we show that in certain parameter regions noise aids the reliability of logic operations, and is actually necessary for obtaining consistent outputs. This leads us to a generalization of the concept of Logical Stochastic Resonance to attractors more complex than fixed point states, such as periodic or chaotic attractors. Lastly, the results are verified in electronic circuit experiments, demonstrating the robustness of the phenomena. So we have combined the research directions of Chaos Computing and Logical Stochastic Resonance here, and this approach has the potential to be realized in wide-ranging systems. △ Less

Submitted 25 November, 2018; originally announced November 2018.

Detecting and characterizing high frequency oscillations in epilepsy - A case study of big data analysis

Authors: Liang Huang, Xuan Ni, William L. Ditto, Mark Spano, Paul R. Carney, Ying-Cheng Lai

Abstract: We develop a framework to uncover and analyze dynamical anomalies from massive, nonlinear and non-stationary time series data. The framework consists of three steps: preprocessing of massive data sets to eliminate erroneous data segments, application of the empirical mode decomposition and Hilbert transform paradigm to obtain the fundamental components embedded in the time series at distinct time… ▽ More We develop a framework to uncover and analyze dynamical anomalies from massive, nonlinear and non-stationary time series data. The framework consists of three steps: preprocessing of massive data sets to eliminate erroneous data segments, application of the empirical mode decomposition and Hilbert transform paradigm to obtain the fundamental components embedded in the time series at distinct time scales, and statistical/scaling analysis of the components. As a case study, we apply our framework to detecting and characterizing high frequency oscillations (HFOs) from a big database of rat EEG recordings. We find a striking phenomenon: HFOs exhibit on-off intermittency that can be quantified by algebraic scaling laws. Our framework can be generalized to big data-related problems in other fields such as large-scale sensor data and seismic data analysis. △ Less

Submitted 22 December, 2016; originally announced December 2016.

Comments: 22 pages, 11 figures

Simple nonlinear models suggest variable star universality

Authors: John F. Lindner, Vivek Kohar, Behnam Kia, Michael Hippke, John G. Learned, William L. Ditto

Abstract: Dramatically improved data from observatories like the CoRoT and Kepler spacecraft have recently facilitated nonlinear time series analysis and phenomenological modeling of variable stars, including the search for strange (aka fractal) or chaotic dynamics. We recently argued [Lindner et al., Phys. Rev. Lett. 114 (2015) 054101] that the Kepler data includes "golden" stars, whose luminosities vary q… ▽ More Dramatically improved data from observatories like the CoRoT and Kepler spacecraft have recently facilitated nonlinear time series analysis and phenomenological modeling of variable stars, including the search for strange (aka fractal) or chaotic dynamics. We recently argued [Lindner et al., Phys. Rev. Lett. 114 (2015) 054101] that the Kepler data includes "golden" stars, whose luminosities vary quasiperiodically with two frequencies nearly in the golden ratio, and whose secondary frequencies exhibit power-law scaling with exponent near -1.5, suggesting strange nonchaotic dynamics and singular spectra. Here we use a series of phenomenological models to make plausible the connection between golden stars and fractal spectra. We thereby suggest that at least some features of variable star dynamics reflect universal nonlinear phenomena common to even simple systems. △ Less

Submitted 15 October, 2015; originally announced October 2015.

Comments: 9 pages, 9 figures, accepted for publication in Physica D

Strange nonchaotic stars

Authors: John F. Lindner, Vivek Kohar, Behnam Kia, Michael Hippke, John G. Learned, William L. Ditto

Abstract: The unprecedented light curves of the Kepler space telescope document how the brightness of some stars pulsates at primary and secondary frequencies whose ratios are near the golden mean, the most irrational number. A nonlinear dynamical system driven by an irrational ratio of frequencies generically exhibits a strange but nonchaotic attractor. For Kepler's "golden" stars, we present evidence of t… ▽ More The unprecedented light curves of the Kepler space telescope document how the brightness of some stars pulsates at primary and secondary frequencies whose ratios are near the golden mean, the most irrational number. A nonlinear dynamical system driven by an irrational ratio of frequencies generically exhibits a strange but nonchaotic attractor. For Kepler's "golden" stars, we present evidence of the first observation of strange nonchaotic dynamics in nature outside the laboratory. This discovery could aid the classification and detailed modeling of variable stars. △ Less

Submitted 4 February, 2015; v1 submitted 8 January, 2015; originally announced January 2015.

Comments: 5 pages, 4 figures, published in Physical Review Letters

Journal ref: Phys. Rev. Lett. 114, 054101 (2015)

Coupling Reduces Noise

Authors: Behnam Kia, Sarvenaz Kia, John F. Lindner, Sudeshna Sinha, William L. Ditto

Abstract: We demonstrate how coupling nonlinear dynamical systems can reduce the effects of noise. For simplicity we investigate noisy coupled map lattices. Noise from different lattice nodes can diffuse across the lattice and lower the noise level of individual nodes. We develop a theoretical model that explains this observed noise evolution and show how the coupled dynamics can naturally function as an av… ▽ More We demonstrate how coupling nonlinear dynamical systems can reduce the effects of noise. For simplicity we investigate noisy coupled map lattices. Noise from different lattice nodes can diffuse across the lattice and lower the noise level of individual nodes. We develop a theoretical model that explains this observed noise evolution and show how the coupled dynamics can naturally function as an averaging filter. Our numerical simulations are in excellent agreement with the model predictions. △ Less

Submitted 18 September, 2014; originally announced September 2014.

Journal ref: International Journal of Bifurcation and Chaos 25.03 (2015): 1550040

Pulsation period variations in the RRc Lyrae star KIC 5520878

Authors: Michael Hippke, John G. Learned, A. Zee, William H. Edmondson, Ian R. Steven, John F. Lindner, Benham Kia, William L. Ditto

Abstract: Learned et. al. proposed that a sufficiently advanced extra-terrestrial civilization may tickle Cepheid and RR Lyrae variable stars with a neutrino beam at the right time, thus causing them to trigger early and jogging the otherwise very regular phase of their expansion and contraction. This would turn these stars into beacons to transmit information throughout the galaxy and beyond. The idea is t… ▽ More Learned et. al. proposed that a sufficiently advanced extra-terrestrial civilization may tickle Cepheid and RR Lyrae variable stars with a neutrino beam at the right time, thus causing them to trigger early and jogging the otherwise very regular phase of their expansion and contraction. This would turn these stars into beacons to transmit information throughout the galaxy and beyond. The idea is to search for signs of phase modulation (in the regime of short pulse duration) and patterns, which could be indicative of intentional, omnidirectional signaling. We have performed such a search among variable stars using photometric data from the Kepler space telescope. In the RRc Lyrae star KIC 5520878, we have found two such regimes of long and short pulse durations. The sequence of period lengths, expressed as time series data, is strongly auto correlated, with correlation coefficients of prime numbers being significantly higher ($p=99.8$\%). Our analysis of this candidate star shows that the prime number oddity originates from two simultaneous pulsation periods and is likely of natural origin. Simple physical models elucidate the frequency content and asymmetries of the KIC 5520878 light curve. Despite this SETI null result, we encourage testing other archival and future time-series photometry for signs of modulated stars. This can be done as a by-product to the standard analysis, and even partly automated. △ Less

Submitted 23 October, 2014; v1 submitted 3 September, 2014; originally announced September 2014.

Comments: Accepted for publication in ApJ. 49 pages, 16 figures

Journal ref: ApJ, 798, 42 (2015)

Synchrony with Shunting Inhibition

Authors: Sachin S. Talathi, Dong-Uk Hwang, Abraham Miliotis, Paul R. Carney, William L. Ditto

Abstract: Spike time response curves (STRC's) are used to study the influence of synaptic stimuli on the firing times of a neuron oscillator without the assumption of weak coupling. They allow us to approximate the dynamics of synchronous state in networks of neurons through a discrete map. Linearization about the fixed point of the discrete map can then be used to predict the stability of patterns of syn… ▽ More Spike time response curves (STRC's) are used to study the influence of synaptic stimuli on the firing times of a neuron oscillator without the assumption of weak coupling. They allow us to approximate the dynamics of synchronous state in networks of neurons through a discrete map. Linearization about the fixed point of the discrete map can then be used to predict the stability of patterns of synchrony in the network. General theory for taking into account the contribution from higher order STRC terms, in the approximation of the discrete map for coupled neuronal oscillators in synchrony is still lacking. Here we present a general framework to account for higher order STRC corrections in the approximation of discrete map to determine the domain of 1:1 phase locking state in the network of two interacting neurons. We begin by demonstrating that the effect of synaptic stimuli through a shunting synapse to a neuron firing in the gamma frequency band (20-80 Hz) last for three consecutive firing cycles. We then show that the discrete map derived by taking into account the higher order STRC contributions is successfully able predict the domain of synchronous 1:1 phase locked state in a network of two heterogeneous interneurons coupled through a shunting synapse. △ Less

Submitted 30 January, 2009; v1 submitted 26 January, 2009; originally announced January 2009.

Comments: 4 Figures

Flexible Logic from Neuronal Dynamics

Authors: Abraham Miliotis, Sachin S. Talathi, William L. Ditto

Abstract: We present two novel methods for performing logic operations. Our methods are based on using the time dimension for programming and data representation. The first method is based on varying the sampling moment in time of a neuronal action potential, and the second method is based on a neural delay system, where the generation of the action potential is delayed by specific time lengths, to be sam… ▽ More We present two novel methods for performing logic operations. Our methods are based on using the time dimension for programming and data representation. The first method is based on varying the sampling moment in time of a neuronal action potential, and the second method is based on a neural delay system, where the generation of the action potential is delayed by specific time lengths, to be sampled at a fixed moment in time. Both methods are supported by explicit examples. △ Less

Submitted 3 June, 2008; originally announced June 2008.

Comments: 5 pages, 5 Figures

Noise sustained propagation: Local versus global noise

Authors: M. Locher, N. Chatterjee, F. Marchesoni, W. L. Ditto, E. R. Hunt

Abstract: We expand on prior results on noise supported signal propagation in arrays of coupled bistable elements. We present and compare experimental and numerical results for kink propagation under the influence of local and global fluctuations. As demonstrated previously for local noise, an optimum range of global noise power exists for which the medium acts as a reliable transmission ``channel''. We d… ▽ More We expand on prior results on noise supported signal propagation in arrays of coupled bistable elements. We present and compare experimental and numerical results for kink propagation under the influence of local and global fluctuations. As demonstrated previously for local noise, an optimum range of global noise power exists for which the medium acts as a reliable transmission ``channel''. We discuss implications for propagation failure in a model of cardiac tissue and present a general theoretical framework based on discrete kink statistics. Valid for generic bistable chains, the theory captures the essential features ob served in our experiments and numerical simulations. △ Less

Submitted 23 March, 2000; originally announced March 2000.

Comments: 1 latex file 20 pages, 9 figures. Accepted for publication in Physical Review E