-
Optimization with delay-induced bifurcations
Authors:
Natalia B. Janson,
Christopher J. Marsden
Abstract:
Optimization is finding the best solution, which mathematically amounts to locating the global minimum of some cost function. Optimization is traditionally automated with digital or quantum computers, each having their limitations and none guaranteeing an optimal solution. Here, we conceive a principle behind optimization based on delay-induced bifurcations, which is potentially implementable in n…
▽ More
Optimization is finding the best solution, which mathematically amounts to locating the global minimum of some cost function. Optimization is traditionally automated with digital or quantum computers, each having their limitations and none guaranteeing an optimal solution. Here, we conceive a principle behind optimization based on delay-induced bifurcations, which is potentially implementable in non-quantum analog devices. Often, optimization techniques are interpreted via a particle moving in multi-well energy landscape and to prevent confinement to a non-global minima they should incorporate mechanisms to overcome barriers between the minima. Particularly, simulated annealing digitally emulates pushing a fictitious particle over a barrier by random noise, whereas quantum computers utilize tunneling through barriers. In our principle, the barriers are effectively destroyed by delay-induced bifurcations. Although bifurcation scenarios in nonlinear delay-differential equations can be very complex and are notoriously difficult to predict, we hypothesize, verify, and utilize the finding that they become considerably more predictable in dynamical systems, where the right-hand side depends only on the delayed variable and represents a gradient of some potential energy function. By tuning the delay introduced into the gradient descent setting, thanks to global bifurcations destroying local attractors, one could force the system to spontaneously wander around all minima. This would be similar to noise-induced behavior in simulated annealing but achieved deterministically. Ideally, a slow increase and then decrease of the delay should automatically push the system toward the global minimum. We explore the possibility of this scenario and formulate some prerequisites.
△ Less
Submitted 13 November, 2021; v1 submitted 21 February, 2019;
originally announced February 2019.
-
Delay-induced homoclinic bifurcations in modified gradient bistable systems and their relevance to optimisation
Authors:
Natalia B. Janson,
Christopher J. Marsden
Abstract:
Nonlinear dynamical systems with time delay are abundant in applications, but are notoriously difficult to analyse and predict because delay-induced effects strongly depend on the form of the nonlinearities involved, and on the exact way the delay enters the system. We consider a special class of nonlinear systems with delay obtained by taking a gradient dynamical system with a two-well "potential…
▽ More
Nonlinear dynamical systems with time delay are abundant in applications, but are notoriously difficult to analyse and predict because delay-induced effects strongly depend on the form of the nonlinearities involved, and on the exact way the delay enters the system. We consider a special class of nonlinear systems with delay obtained by taking a gradient dynamical system with a two-well "potential" function and replacing the argument of the right-hand side function with its delayed version. This choice of the system is motivated by the relative ease of its graphical interpretation, and by its relevance to a recent approach to use delay in finding the global minimum of a multi-well function. Here, the simplest type of such systems is explored, for which we hypothesise and verify the possibility to qualitatively predict the delay-induced effects, such as a chain of homoclinic bifurcations one by one eliminating local attractors and enabling the phase trajectory to spontaneously visit vicinities of all local minima. The key phenomenon here is delay-induced reorganisation of manifolds, which cease to serve as barriers between the local minima after homoclinic bifurcations. Despite the general scenario being quite universal in two-well potentials, the homoclinic bifurcation comes in various versions depending on the fine features of the potential. Our results are a pre-requisite for understanding general highly nonlinear multistable systems with delay. They also reveal the mechanisms behind the possible role of delay in optimisation.
△ Less
Submitted 1 November, 2021; v1 submitted 17 February, 2019;
originally announced February 2019.
-
Conceptual model of a cognitive system: dynamical system with plastic self-organizing velocity field
Authors:
Natalia B. Janson,
Christopher J. Marsden
Abstract:
Spontaneously evolving living systems can be modelled as continuous-time dynamical systems (DSs), whose evolution rules are determined by their velocity vector fields. We point out that because of their architectural plasticity, biological neural networks belong to a novel type of DSs whose velocity field is plastic, albeit within bounds, and affected by sensory stimuli. We introduce DSs with full…
▽ More
Spontaneously evolving living systems can be modelled as continuous-time dynamical systems (DSs), whose evolution rules are determined by their velocity vector fields. We point out that because of their architectural plasticity, biological neural networks belong to a novel type of DSs whose velocity field is plastic, albeit within bounds, and affected by sensory stimuli. We introduce DSs with fully plastic velocity fields self-organising under the influence of stimuli, called self-shaping DSs, and propose that a system of this class represents a conceptual model of a cognitive system. We propose a simple phenomenological model that within a single field-shaping mechanism carries out a set of essential cognitive functions without any supervision and online, just like living cognitive systems do. The performance of this model is illustrated experimentally with musical examples. Unlike in artificial neural networks, this mechanism does not produce spurious attractors and in principle does not limit memory size. Implementation of this principle could pave the way to creating artificial intelligent devices of a new type.
△ Less
Submitted 6 February, 2016; v1 submitted 31 January, 2016;
originally announced February 2016.
-
Self-shaping dynamical systems and learning
Authors:
Natalia B. Janson,
Christopher J. Marsden
Abstract:
We associate learning and adaptation in living systems with the shaping of the velocity vector field in the respective dynamical systems in response to external, generally random, stimuli. With this, a mathematical concept of self-shaping dynamical systems is proposed. Initially there is a zero vector field and an "empty" phase space with no attractors or other non-trivial objects. As the random s…
▽ More
We associate learning and adaptation in living systems with the shaping of the velocity vector field in the respective dynamical systems in response to external, generally random, stimuli. With this, a mathematical concept of self-shaping dynamical systems is proposed. Initially there is a zero vector field and an "empty" phase space with no attractors or other non-trivial objects. As the random stimulus begins, the vector field deforms and eventually becomes smooth and deterministic, despite the random nature of the applied force, while the phase space develops various geometrical objects. We consider gradient self-shaping systems, whose vector field is the gradient of some energy function, which under certain conditions develops into the multi-dimensional probability density distribution (PDD) of the input. Self-shaping systems are relevant to neural networks (NNs) of two types: Hopfield, and probabilistic. Firstly, we show that they can potentially perform pattern recognition tasks traditionally delegated to Hopfield NNs, but without supervision and on-line, and without developing spurious minima of the energy. Secondly, like probabilistic NNs, they can reconstruct the PDD of input signals, without the limitation that new training patterns have to enter as new hardware units. Thus, self-shaping systems can be regarded as a generalization of the NN concept, achieved by abandoning the "rigid units" - "flexible couplings" paradigm and making the vector field fully flexible and amenable to external force. The new concept presents an engineering challenge requiring new principles of hardware design. It might also become an alternative paradigm for modeling of living and learning systems.
△ Less
Submitted 18 November, 2011;
originally announced November 2011.
-
"Memory foam" approach to unsupervised learning
Authors:
Natalia B. Janson,
Christopher J. Marsden
Abstract:
We propose an alternative approach to construct an artificial learning system, which naturally learns in an unsupervised manner. Its mathematical prototype is a dynamical system, which automatically shapes its vector field in response to the input signal. The vector field converges to a gradient of a multi-dimensional probability density distribution of the input process, taken with negative sign.…
▽ More
We propose an alternative approach to construct an artificial learning system, which naturally learns in an unsupervised manner. Its mathematical prototype is a dynamical system, which automatically shapes its vector field in response to the input signal. The vector field converges to a gradient of a multi-dimensional probability density distribution of the input process, taken with negative sign. The most probable patterns are represented by the stable fixed points, whose basins of attraction are formed automatically. The performance of this system is illustrated with musical signals.
△ Less
Submitted 7 October, 2015; v1 submitted 4 July, 2011;
originally announced July 2011.
-
Noise-induced cooperative dynamics and its control in coupled neuron models
Authors:
B. Hauschildt,
N. B. Janson,
A. Balanov,
E. Schoell
Abstract:
We investigate feedback control of the cooperative dynamics of two coupled neural oscillators that is induced merely by external noise. The interacting neurons are modelled as FitzHugh-Nagumo systems with parameter values at which no autonomous oscillations occur, and each unit is forced by its own source of random fluctuations. Application of delayed feedback to only one of two subsystems is sh…
▽ More
We investigate feedback control of the cooperative dynamics of two coupled neural oscillators that is induced merely by external noise. The interacting neurons are modelled as FitzHugh-Nagumo systems with parameter values at which no autonomous oscillations occur, and each unit is forced by its own source of random fluctuations. Application of delayed feedback to only one of two subsystems is shown to be able to change coherence and timescales of noise induced oscillations either in the given subsystem, or globally. It is also able to induce or to suppress stochastic synchronization under certain conditions.
△ Less
Submitted 4 October, 2006;
originally announced October 2006.
