Showing 1–27 of 27 results for author: Nakajima, K

Attractor-merging Crises and Intermittency in Reservoir Computing

Authors: Tempei Kabayama, Motomasa Komuro, Yasuo Kuniyoshi, Kazuyuki Aihara, Kohei Nakajima

Abstract: Reservoir computing can embed attractors into random neural networks (RNNs), generating a ``mirror'' of a target attractor because of its inherent symmetrical constraints. In these RNNs, we report that an attractor-merging crisis accompanied by intermittency emerges simply by adjusting the global parameter. We further reveal its underlying mechanism through a detailed analysis of the phase-space s… ▽ More Reservoir computing can embed attractors into random neural networks (RNNs), generating a ``mirror'' of a target attractor because of its inherent symmetrical constraints. In these RNNs, we report that an attractor-merging crisis accompanied by intermittency emerges simply by adjusting the global parameter. We further reveal its underlying mechanism through a detailed analysis of the phase-space structure and demonstrate that this bifurcation scenario is intrinsic to a general class of RNNs, independent of training data. △ Less

Submitted 17 April, 2025; originally announced April 2025.

Comments: 20 pages, 15 figures

Harnessing omnipresent oscillator networks as computational resource

Authors: Thomas Geert de Jong, Hirofumi Notsu, Kohei Nakajima

Abstract: Nature is pervaded with oscillatory behavior. In networks of coupled oscillators patterns can arise when the system synchronizes to an external input. Hence, these networks provide processing and memory of input. We present a universal framework for harnessing oscillator networks as computational resource. This reservoir computing framework is introduced by the ubiquitous model for phase-locking,… ▽ More Nature is pervaded with oscillatory behavior. In networks of coupled oscillators patterns can arise when the system synchronizes to an external input. Hence, these networks provide processing and memory of input. We present a universal framework for harnessing oscillator networks as computational resource. This reservoir computing framework is introduced by the ubiquitous model for phase-locking, the Kuramoto model. We force the Kuramoto model by a nonlinear target-system, then after substituting the target-system with a trained feedback-loop it emulates the target-system. Our results are two-fold. Firstly, the trained network inherits performance properties of the Kuramoto model, where all-to-all coupling is performed in linear time with respect to the number of nodes and parameters for synchronization are abundant. Secondly, the learning capabilities of the oscillator network can be explained using Kuramoto model's order parameter. This work provides the foundation for utilizing nature's oscillator networks as a new class of information processing systems. △ Less

Submitted 21 February, 2025; v1 submitted 7 February, 2025; originally announced February 2025.

MSC Class: 92B25 ACM Class: I.2.6

Reservoir Computing Generalized

Authors: Tomoyuki Kubota, Yusuke Imai, Sumito Tsunegi, Kohei Nakajima

Abstract: A physical neural network (PNN) has both the strong potential to solve machine learning tasks and intrinsic physical properties, such as high-speed computation and energy efficiency. Reservoir computing (RC) is an excellent framework for implementing an information processing system with a dynamical system by attaching a trained readout, thus accelerating the wide use of unconventional materials f… ▽ More A physical neural network (PNN) has both the strong potential to solve machine learning tasks and intrinsic physical properties, such as high-speed computation and energy efficiency. Reservoir computing (RC) is an excellent framework for implementing an information processing system with a dynamical system by attaching a trained readout, thus accelerating the wide use of unconventional materials for a PNN. However, RC requires the dynamics to reproducibly respond to input sequence, which limits the type of substance available for building information processors. Here we propose a novel framework called generalized reservoir computing (GRC) by turning this requirement on its head, making conventional RC a special case. Using substances that do not respond the same to identical inputs (e.g., a real spin-torque oscillator), we propose mechanisms aimed at obtaining a reliable output and show that processed inputs in the unconventional substance are retrievable. Finally, we demonstrate that, based on our framework, spatiotemporal chaos, which is thought to be unusable as a computational resource, can be used to emulate complex nonlinear dynamics, including large scale spatiotemporal chaos. Overall, our framework removes the limitation to building an information processing device and opens a path to constructing a computational system using a wider variety of physical dynamics. △ Less

Submitted 23 November, 2024; originally announced December 2024.

Designing Chaotic Attractors: A Semi-supervised Approach

Authors: Tempei Kabayama, Yasuo Kuniyoshi, Kazuyuki Aihara, Kohei Nakajima

Abstract: Chaotic dynamics are ubiquitous in nature and useful in engineering, but their geometric design can be challenging. Here, we propose a method using reservoir computing to generate chaos with a desired shape by providing a periodic orbit as a template, called a skeleton. We exploit a bifurcation of the reservoir to intentionally induce unsuccessful training of the skeleton, revealing inherent chaos… ▽ More Chaotic dynamics are ubiquitous in nature and useful in engineering, but their geometric design can be challenging. Here, we propose a method using reservoir computing to generate chaos with a desired shape by providing a periodic orbit as a template, called a skeleton. We exploit a bifurcation of the reservoir to intentionally induce unsuccessful training of the skeleton, revealing inherent chaos. The emergence of this untrained attractor, resulting from the interaction between the skeleton and the reservoir's intrinsic dynamics, offers a novel semi-supervised framework for designing chaos. △ Less

Submitted 27 June, 2024; originally announced July 2024.

Comments: 6 pages, 4 figures (excluding supplementary material)

Thermodynamic limit in learning period three

Authors: Yuichiro Terasaki, Kohei Nakajima

Abstract: A continuous one-dimensional map with period three includes all periods. This raises the following question: Can we obtain any types of periodic orbits solely by learning three data points? In this paper, we report the answer to be yes. Considering a random neural network in its thermodynamic limit, we first show that almost all learned periods are unstable, and each network has its own characteri… ▽ More A continuous one-dimensional map with period three includes all periods. This raises the following question: Can we obtain any types of periodic orbits solely by learning three data points? In this paper, we report the answer to be yes. Considering a random neural network in its thermodynamic limit, we first show that almost all learned periods are unstable, and each network has its own characteristic attractors (which can even be untrained ones). The latently acquired dynamics, which are unstable within the trained network, serve as a foundation for the diversity of characteristic attractors and may even lead to the emergence of attractors of all periods after learning. When the neural network interpolation is quadratic, a universal post-learning bifurcation scenario appears, which is consistent with a topological conjugacy between the trained network and the classical logistic map. In addition to universality, we explore specific properties of certain networks, including the singular behavior of the scale of weight at the infinite limit, the finite-size effects, and the symmetry in learning period three. △ Less

Submitted 14 April, 2025; v1 submitted 12 May, 2024; originally announced May 2024.

Comments: 19 pages, 12 figures

Input-driven chaotic dynamics in vortex spin-torque oscillator

Authors: Yusuke Imai, Kohei Nakajima, Sumito Tsunegi, Tomohiro Taniguchi

Abstract: A new research topic in spintronics relating to the operation principles of brain-inspired computing is input-driven magnetization dynamics in nanomagnet. In this paper, the magnetization dynamics in a vortex spin-torque oscillator (STO) driven by a series of random magnetic field are studied through a numerical simulation of the Thiele equation. It is found that input-driven synchronization occur… ▽ More A new research topic in spintronics relating to the operation principles of brain-inspired computing is input-driven magnetization dynamics in nanomagnet. In this paper, the magnetization dynamics in a vortex spin-torque oscillator (STO) driven by a series of random magnetic field are studied through a numerical simulation of the Thiele equation. It is found that input-driven synchronization occurs in the weak perturbation limit, as found recently. As well, chaotic behavior is newly found to occur in the vortex core dynamics for a wide range of parameters, where synchronized behavior is disrupted by an intermittency. Ordered and chaotic dynamical phases are examined by evaluating the Lyapunov exponent. The relation between the dynamical phase and the computational capability of physical reservoir computing is also studied. △ Less

Submitted 22 June, 2023; originally announced June 2023.

Comments: 11 pages, 4 figures

Journal ref: Scientific Reports 12, 21651 (2022)

Embedding bifurcations into pneumatic artificial muscle

Authors: Nozomi Akashi, Yasuo Kuniyoshi, Taketomo Jo, Mitsuhiro Nishida, Ryo Sakurai, Yasumichi Wakao, Kohei Nakajima

Abstract: Harnessing complex body dynamics has been a long-standing challenge in robotics. Soft body dynamics is a typical example of high complexity in interacting with the environment. An increasing number of studies have reported that these dynamics can be used as a computational resource. This includes the McKibben pneumatic artificial muscle, which is a typical soft actuator. This study demonstrated th… ▽ More Harnessing complex body dynamics has been a long-standing challenge in robotics. Soft body dynamics is a typical example of high complexity in interacting with the environment. An increasing number of studies have reported that these dynamics can be used as a computational resource. This includes the McKibben pneumatic artificial muscle, which is a typical soft actuator. This study demonstrated that various dynamics, including periodic and chaotic dynamics, could be embedded into the pneumatic artificial muscle, with the entire bifurcation structure using the framework of physical reservoir computing. These results suggest that dynamics that are not presented in training data could be embedded by using this capability of bifurcation embeddment. This implies that it is possible to embed various qualitatively different patterns into pneumatic artificial muscle by learning specific patterns, without the need to design and learn all patterns required for the purpose. Thus, this study sheds new light on a novel pathway to simplify the robotic devices and training of the control by reducing the external pattern generators and the amount and types of training data for the control. △ Less

Submitted 10 May, 2023; v1 submitted 6 May, 2023; originally announced May 2023.

ACM Class: I.2.9

Computational capability for physical reservoir computing using a spin-torque oscillator with two free layers

Authors: Terufumi Yamaguchi, Sumito Tsunegi, Kohei Nakajima, Tomohiro Taniguchi

Abstract: A numerical analysis on the computational capability of physical reservoir computing utilizing a spin-torque oscillator with two free layers is reported. Conventional spintronics devices usually consist of two ferromagnets, where the direction of magnetization in one layer, called the free layer, can move while that of the other, the reference layer, is fixed. Recently, however, devices with two f… ▽ More A numerical analysis on the computational capability of physical reservoir computing utilizing a spin-torque oscillator with two free layers is reported. Conventional spintronics devices usually consist of two ferromagnets, where the direction of magnetization in one layer, called the free layer, can move while that of the other, the reference layer, is fixed. Recently, however, devices with two free layers, where the reference layer is replaced by another free layer, have been developed for various practical applications. Adding another free layer drastically changes the dynamical response of the device through the couplings via the spin-transfer effect and the dipole magnetic field. A numerical simulation of the Landau-Lifshitz-Gilbert equation and a statistical analyses of the Lyapunov exponent and the synchronization index reveal the appearance of an amplitude-modulated oscillation and chaos in the oscillators with two free layers. Such complex dynamics qualitatively change the computational capability of physical reservoir computing because the computational resource is dynamics of the physical system. An evaluation of the short-term memory capacity clarifies that oscillators with two free layers have a larger capacity than those of conventional oscillators. An enhancement in capacity near the edge of echo state property, i.e., the boundary between zero and finite synchronization index, is also found. △ Less

Submitted 7 February, 2023; originally announced February 2023.

Comments: 16 pages, 11 figures

Journal ref: Phys. Rev. B 107, 054406 (2023)

Physical Deep Learning with Biologically Plausible Training Method

Authors: Mitsumasa Nakajima, Katsuma Inoue, Kenji Tanaka, Yasuo Kuniyoshi, Toshikazu Hashimoto, Kohei Nakajima

Abstract: The ever-growing demand for further advances in artificial intelligence motivated research on unconventional computation based on analog physical devices. While such computation devices mimic brain-inspired analog information processing, learning procedures still relies on methods optimized for digital processing such as backpropagation. Here, we present physical deep learning by extending a biolo… ▽ More The ever-growing demand for further advances in artificial intelligence motivated research on unconventional computation based on analog physical devices. While such computation devices mimic brain-inspired analog information processing, learning procedures still relies on methods optimized for digital processing such as backpropagation. Here, we present physical deep learning by extending a biologically plausible training algorithm called direct feedback alignment. As the proposed method is based on random projection with arbitrary nonlinear activation, we can train a physical neural network without knowledge about the physical system. In addition, we can emulate and accelerate the computation for this training on a simple and scalable physical system. We demonstrate the proof-of-concept using a hierarchically connected optoelectronic recurrent neural network called deep reservoir computer. By constructing an FPGA-assisted optoelectronic benchtop, we confirmed the potential for accelerated computation with competitive performance on benchmarks. Our results provide practical solutions for the training and acceleration of neuromorphic computation. △ Less

Submitted 1 April, 2022; originally announced April 2022.

Transient Chaos in BERT

Authors: Katsuma Inoue, Soh Ohara, Yasuo Kuniyoshi, Kohei Nakajima

Abstract: Language is an outcome of our complex and dynamic human-interactions and the technique of natural language processing (NLP) is hence built on human linguistic activities. Bidirectional Encoder Representations from Transformers (BERT) has recently gained its popularity by establishing the state-of-the-art scores in several NLP benchmarks. A Lite BERT (ALBERT) is literally characterized as a lightwe… ▽ More Language is an outcome of our complex and dynamic human-interactions and the technique of natural language processing (NLP) is hence built on human linguistic activities. Bidirectional Encoder Representations from Transformers (BERT) has recently gained its popularity by establishing the state-of-the-art scores in several NLP benchmarks. A Lite BERT (ALBERT) is literally characterized as a lightweight version of BERT, in which the number of BERT parameters is reduced by repeatedly applying the same neural network called Transformer's encoder layer. By pre-training the parameters with a massive amount of natural language data, ALBERT can convert input sentences into versatile high-dimensional vectors potentially capable of solving multiple NLP tasks. In that sense, ALBERT can be regarded as a well-designed high-dimensional dynamical system whose operator is the Transformer's encoder, and essential structures of human language are thus expected to be encapsulated in its dynamics. In this study, we investigated the embedded properties of ALBERT to reveal how NLP tasks are effectively solved by exploiting its dynamics. We thereby aimed to explore the nature of human language from the dynamical expressions of the NLP model. Our short-term analysis clarified that the pre-trained model stably yields trajectories with higher dimensionality, which would enhance the expressive capacity required for NLP tasks. Also, our long-term analysis revealed that ALBERT intrinsically shows transient chaos, a typical nonlinear phenomenon showing chaotic dynamics only in its transient, and the pre-trained ALBERT model tends to produce the chaotic trajectory for a significantly longer time period compared to a randomly-initialized one. Our results imply that local chaoticity would contribute to improving NLP performance, uncovering a novel aspect in the role of chaotic dynamics in human language behaviors. △ Less

Submitted 8 June, 2021; v1 submitted 6 June, 2021; originally announced June 2021.

Comments: 11 pages, 5 figures

Quantum reservoir computing: a reservoir approach toward quantum machine learning on near-term quantum devices

Authors: Keisuke Fujii, Kohei Nakajima

Abstract: Quantum systems have an exponentially large degree of freedom in the number of particles and hence provide a rich dynamics that could not be simulated on conventional computers. Quantum reservoir computing is an approach to use such a complex and rich dynamics on the quantum systems as it is for temporal machine learning. In this chapter, we explain quantum reservoir computing and related approach… ▽ More Quantum systems have an exponentially large degree of freedom in the number of particles and hence provide a rich dynamics that could not be simulated on conventional computers. Quantum reservoir computing is an approach to use such a complex and rich dynamics on the quantum systems as it is for temporal machine learning. In this chapter, we explain quantum reservoir computing and related approaches, quantum extreme learning machine and quantum circuit learning, starting from a pedagogical introduction to quantum mechanics and machine learning. All these quantum machine learning approaches are experimentally feasible and effective on the state-of-the-art quantum devices. △ Less

Submitted 9 November, 2020; originally announced November 2020.

Comments: 13 pages, 4 figures

Periodic structure of memory function in spintronics reservoir with feedback current

Authors: Terufumi Yamaguchi, Nozomi Akashi, Sumito Tsunegi, Hitoshi Kubota, Kohei Nakajima, Tomohiro Taniguchi

Abstract: The role of the feedback effect on physical reservoir computing is studied theoretically by solving the vortex-core dynamics in a nanostructured ferromagnet. Although the spin-transfer torque due to the feedback current makes the vortex dynamics complex, it is clarified that the feedback effect does not always contribute to the enhancement of the memory function in a physical reservoir. The memory… ▽ More The role of the feedback effect on physical reservoir computing is studied theoretically by solving the vortex-core dynamics in a nanostructured ferromagnet. Although the spin-transfer torque due to the feedback current makes the vortex dynamics complex, it is clarified that the feedback effect does not always contribute to the enhancement of the memory function in a physical reservoir. The memory function, characterized by the correlation coefficient between the input data and the dynamical response of the vortex core, becomes large when the delay time of the feedback current is not an integral multiple of the pulse width. On the other hand, the memory function remains small when the delay time is an integral multiple of the pulse width. As a result, a periodic behavior for the short-term memory capacity is observed with respect to the delay time, the phenomenon of which can be attributed to correlations between the virtual neurons via the feedback current. △ Less

Submitted 23 June, 2020; originally announced June 2020.

Comments: 10 pages, 9 figures

Journal ref: Phys. Rev. Research 2, 023389 (2020)

Higher-Order Quantum Reservoir Computing

Authors: Quoc Hoan Tran, Kohei Nakajima

Abstract: Quantum reservoir computing (QRC) is an emerging paradigm for harnessing the natural dynamics of quantum systems as computational resources that can be used for temporal machine learning tasks. In the current setup, QRC is difficult to deal with high-dimensional data and has a major drawback of scalability in physical implementations. We propose higher-order QRC, a hybrid quantum-classical framewo… ▽ More Quantum reservoir computing (QRC) is an emerging paradigm for harnessing the natural dynamics of quantum systems as computational resources that can be used for temporal machine learning tasks. In the current setup, QRC is difficult to deal with high-dimensional data and has a major drawback of scalability in physical implementations. We propose higher-order QRC, a hybrid quantum-classical framework consisting of multiple but small quantum systems that are mutually communicated via classical connections like linear feedback. By utilizing the advantages of both classical and quantum techniques, our framework enables an efficient implementation to boost the scalability and performance of QRC. Furthermore, higher-order settings allow us to implement a FORCE learning or an innate training scheme, which provides flexibility and high operability to harness high-dimensional quantum dynamics and significantly extends the application domain of QRC. We demonstrate the effectiveness of our framework in emulating large-scale nonlinear dynamical systems, including complex spatiotemporal chaos, which outperforms many of the existing machine learning techniques in certain situations. △ Less

Submitted 20 October, 2020; v1 submitted 16 June, 2020; originally announced June 2020.

Comments: main (15 pages, 12 pictures), add "Quantum-innate training"

Physical reservoir computing -- An introductory perspective

Authors: Kohei Nakajima

Abstract: Understanding the fundamental relationships between physics and its information-processing capability has been an active research topic for many years. Physical reservoir computing is a recently introduced framework that allows one to exploit the complex dynamics of physical systems as information-processing devices. This framework is particularly suited for edge computing devices, in which inform… ▽ More Understanding the fundamental relationships between physics and its information-processing capability has been an active research topic for many years. Physical reservoir computing is a recently introduced framework that allows one to exploit the complex dynamics of physical systems as information-processing devices. This framework is particularly suited for edge computing devices, in which information processing is incorporated at the edge (e.g., into sensors) in a decentralized manner to reduce the adaptation delay caused by data transmission overhead. This paper aims to illustrate the potentials of the framework using examples from soft robotics and to provide a concise overview focusing on the basic motivations for introducing it, which stem from a number of fields, including machine learning, nonlinear dynamical systems, biological science, materials science, and physics. △ Less

Submitted 3 May, 2020; originally announced May 2020.

Comments: 18 pages, 8 figures

Designing spontaneous behavioral switching via chaotic itinerancy

Authors: Katsuma Inoue, Kohei Nakajima, Yasuo Kuniyoshi

Abstract: Chaotic itinerancy is a frequently observed phenomenon in high-dimensional and nonlinear dynamical systems, and it is characterized by the random transitions among multiple quasi-attractors. Several studies have revealed that chaotic itinerancy has been observed in brain activity, and it is considered to play a critical role in the spontaneous, stable behavior generation of animals. Thus, chaotic… ▽ More Chaotic itinerancy is a frequently observed phenomenon in high-dimensional and nonlinear dynamical systems, and it is characterized by the random transitions among multiple quasi-attractors. Several studies have revealed that chaotic itinerancy has been observed in brain activity, and it is considered to play a critical role in the spontaneous, stable behavior generation of animals. Thus, chaotic itinerancy is a topic of great interest, particularly for neurorobotics researchers who wish to understand and implement autonomous behavioral controls for agents. However, it is generally difficult to gain control over high-dimensional nonlinear dynamical systems. Hence, the implementation of chaotic itinerancy has mainly been accomplished heuristically. In this study, we propose a novel way of implementing chaotic itinerancy reproducibly and at will in a generic high-dimensional chaotic system. In particular, we demonstrate that our method enables us to easily design both the trajectories of quasi-attractors and the transition rules among them simply by adjusting the limited number of system parameters and by utilizing the intrinsic high-dimensional chaos. Finally, we quantitatively discuss the validity and scope of application through the results of several numerical experiments. △ Less

Submitted 13 February, 2020; originally announced February 2020.

Comments: 15 pages, 6 figures and 1 supplementary figure. Our supplementary videos are available in https://drive.google.com/drive/folders/10iB23OMHQfFIRejZstoXMJRpnpm3-3H5?usp=sharing

Optimal short-term memory before the edge of chaos in driven random recurrent networks

Authors: Taichi Haruna, Kohei Nakajima

Abstract: The ability of discrete-time nonlinear recurrent neural networks to store time-varying small input signals is investigated by mean-field theory. The combination of a small input strength and mean-field assumptions makes it possible to derive an approximate expression for the conditional probability density of the state of a neuron given a past input signal. From this conditional probability densit… ▽ More The ability of discrete-time nonlinear recurrent neural networks to store time-varying small input signals is investigated by mean-field theory. The combination of a small input strength and mean-field assumptions makes it possible to derive an approximate expression for the conditional probability density of the state of a neuron given a past input signal. From this conditional probability density, we can analytically calculate short-term memory measures, such as memory capacity, mutual information, and Fisher information, and determine the relationships among these measures, which have not been clarified to date to the best of our knowledge. We show that the network contribution of these short-term memory measures peaks before the edge of chaos, where the dynamics of input-driven networks is stable but corresponding systems without input signals are unstable. △ Less

Submitted 24 December, 2019; originally announced December 2019.

Journal ref: Phys. Rev. E 100 (2019) 062312

Chaos in nanomagnet via feedback current

Authors: Tomohiro Taniguchi, Nozomi Akashi, Hirofumi Notsu, Masato Kimura, Hiroshi Tsukahara, Kohei Nakajima

Abstract: Nonlinear magnetization dynamics excited by spin-transfer effect with feedback current is studied both numerically and analytically. The numerical simulation of the Landau-Lifshitz-Gilbert equation indicates the positive Lyapunov exponent for a certain range of the feedback rate, which identifies the existence of chaos in a nanostructured ferromagnet. Transient behavior from chaotic to steady osci… ▽ More Nonlinear magnetization dynamics excited by spin-transfer effect with feedback current is studied both numerically and analytically. The numerical simulation of the Landau-Lifshitz-Gilbert equation indicates the positive Lyapunov exponent for a certain range of the feedback rate, which identifies the existence of chaos in a nanostructured ferromagnet. Transient behavior from chaotic to steady oscillation is also observed in another range of the feedback parameter. An analytical theory is also developed, which indicates the appearance of multiple attractors in a phase space due to the feedback current. An instantaneous imbalance between the spin-transfer torque and damping torque causes a transition between the attractors, and results in the complex dynamics. △ Less

Submitted 22 November, 2019; v1 submitted 11 September, 2019; originally announced September 2019.

Journal ref: Phys. Rev. B 100, 174425 (2019)

Harnessing disordered quantum dynamics for machine learning

Authors: Keisuke Fujii, Kohei Nakajima

Abstract: Quantum computer has an amazing potential of fast information processing. However, realisation of a digital quantum computer is still a challenging problem requiring highly accurate controls and key application strategies. Here we propose a novel platform, quantum reservoir computing, to solve these issues successfully by exploiting natural quantum dynamics, which is ubiquitous in laboratories now… ▽ More Quantum computer has an amazing potential of fast information processing. However, realisation of a digital quantum computer is still a challenging problem requiring highly accurate controls and key application strategies. Here we propose a novel platform, quantum reservoir computing, to solve these issues successfully by exploiting natural quantum dynamics, which is ubiquitous in laboratories nowadays, for machine learning. In this framework, nonlinear dynamics including classical chaos can be universally emulated in quantum systems. A number of numerical experiments show that quantum systems consisting of at most seven qubits possess computational capabilities comparable to conventional recurrent neural networks of 500 nodes. This discovery opens up a new paradigm for information processing with artificial intelligence powered by quantum physics. △ Less

Submitted 9 November, 2016; v1 submitted 25 February, 2016; originally announced February 2016.

Comments: 19 pages, 13 figures

Journal ref: Phys. Rev. Applied 8, 024030 (2017)

Measuring information transfer in a soft robotic arm

Authors: Kohei Nakajima, Nico Schmidt, Rolf Pfeifer

Abstract: Soft robots can exhibit diverse behaviors with simple types of actuation by partially outsourcing control to the morphological and material properties of their soft bodies, which is made possible by the tight coupling between control, body, and environment. In this paper, we present a method that will quantitatively characterize these diverse spatiotemporal dynamics of a soft body based on the inf… ▽ More Soft robots can exhibit diverse behaviors with simple types of actuation by partially outsourcing control to the morphological and material properties of their soft bodies, which is made possible by the tight coupling between control, body, and environment. In this paper, we present a method that will quantitatively characterize these diverse spatiotemporal dynamics of a soft body based on the information-theoretic approach. In particular, soft bodies have the ability to propagate the effect of actuation through the entire body, with a certain time delay, due to their elasticity. Our goal is to capture this delayed interaction in a quantitative manner based on a measure called momentary information transfer. We extend this measure to soft robotic applications and demonstrate its power using a physical soft robotic platform inspired by the octopus. Our approach is illustrated in two ways. First, we statistically characterize the delayed actuation propagation through the body as a strength of information transfer. Second, we capture this information propagation directly as local information dynamics. As a result, we show that our approach can successfully characterize the spatiotemporal dynamics of the soft robotic platform, explicitly visualizing how information transfers through the entire body with delays. Further extension scenarios of our approach are discussed for soft robotic applications in general. △ Less

Submitted 15 July, 2014; originally announced July 2014.

Comments: 29 pages, 8 figures

Symbolic local information transfer

Authors: Kohei Nakajima, Taichi Haruna

Abstract: Recently, the permutation-information theoretic approach has been used in a broad range of research fields. In particular, in the study of highdimensional dynamical systems, it has been shown that this approach can be effective in characterizing global properties, including the complexity of their spatiotemporal dynamics. Here, we show that this approach can also be applied to reveal local spatiot… ▽ More Recently, the permutation-information theoretic approach has been used in a broad range of research fields. In particular, in the study of highdimensional dynamical systems, it has been shown that this approach can be effective in characterizing global properties, including the complexity of their spatiotemporal dynamics. Here, we show that this approach can also be applied to reveal local spatiotemporal profiles of distributed computations existing at each spatiotemporal point in the system. J. T. Lizier et al. have recently introduced the concept of local information dynamics, which consists of information storage, transfer, and modification. This concept has been intensively studied with regard to cellular automata, and has provided quantitative evidence of several characteristic behaviors observed in the system. In this paper, by focusing on the local information transfer, we demonstrate that the application of the permutation-information theoretic approach, which introduces natural symbolization methods, makes the concept easily extendible to systems that have continuous states. We propose measures called symbolic local transfer entropies, and apply these measures to two test models, the coupled map lattice (CML) system and the Bak-Sneppen model (BS-model), to show their relevance to spatiotemporal systems that have continuous states. △ Less

Submitted 20 June, 2014; originally announced June 2014.

Comments: 20 pages, 7 figures

Exploiting short-term memory in soft body dynamics as a computational resource

Authors: Kohei Nakajima, Tao Li, Helmut Hauser, Rolf Pfeifer

Abstract: Soft materials are not only highly deformable but they also possess rich and diverse body dynamics. Soft body dynamics exhibit a variety of properties, including nonlinearity, elasticity, and potentially infinitely many degrees of freedom. Here we demonstrate that such soft body dynamics can be employed to conduct certain types of computation. Using body dynamics generated from a soft silicone arm… ▽ More Soft materials are not only highly deformable but they also possess rich and diverse body dynamics. Soft body dynamics exhibit a variety of properties, including nonlinearity, elasticity, and potentially infinitely many degrees of freedom. Here we demonstrate that such soft body dynamics can be employed to conduct certain types of computation. Using body dynamics generated from a soft silicone arm, we show that they can be exploited to emulate functions that require memory and to embed robust closed-loop control into the arm. Our results suggest that soft body dynamics have a short-term memory and can serve as a computational resource. This finding paves the way toward exploiting passive body dynamics for control of a large class of underactuated systems. △ Less

Submitted 20 June, 2014; v1 submitted 18 June, 2014; originally announced June 2014.

Comments: 22 pages, 11 figures; email address corrected

Dynamics Underneath Symbols: A Case Study in Autonomous Agents

Authors: Kohei Nakajima

Abstract: Our cognition is structuring the informational layer, consisting of perception, anticipation, and action, and it should also be sustained on a physical basis. In this paper, we aim to explore the relationship between the informational layer and the physical layer from a dynamical systems point of view. As an example, the fluctuation of choice is investigated by using a simulated agent. By setting… ▽ More Our cognition is structuring the informational layer, consisting of perception, anticipation, and action, and it should also be sustained on a physical basis. In this paper, we aim to explore the relationship between the informational layer and the physical layer from a dynamical systems point of view. As an example, the fluctuation of choice is investigated by using a simulated agent. By setting a T-maze, the agent should choose one arm of the maze if a corresponding token is presented. We prepared two types of tokens, corresponding to the left and right arm of the maze. After training the network of the agent to successfully choose the corresponding arm, we presented two tokens simultaneously to the agent and observed its behavior. As a result, we found several behaviors, which are difficult to speculate on from a case in which only a single token is presented to the agent. Detailed analyses and the implications of the model are discussed. △ Less

Submitted 20 June, 2014; v1 submitted 15 June, 2014; originally announced June 2014.

Comments: 15 pages, 9 figures; Color figures

Permutation Complexity and Coupling Measures in Hidden Markov Models

Authors: Taichi Haruna, Kohei Nakajima

Abstract: In [Haruna, T. and Nakajima, K., 2011. Physica D 240, 1370-1377], the authors introduced the duality between values (words) and orderings (permutations) as a basis to discuss the relationship between information theoretic measures for finite-alphabet stationary stochastic processes and their permutation analogues. It has been used to give a simple proof of the equality between the entropy rate and… ▽ More In [Haruna, T. and Nakajima, K., 2011. Physica D 240, 1370-1377], the authors introduced the duality between values (words) and orderings (permutations) as a basis to discuss the relationship between information theoretic measures for finite-alphabet stationary stochastic processes and their permutation analogues. It has been used to give a simple proof of the equality between the entropy rate and the permutation entropy rate for any finite-alphabet stationary stochastic process and show some results on the excess entropy and the transfer entropy for finite-alphabet stationary ergodic Markov processes. In this paper, we extend our previous results to hidden Markov models and show the equalities between various information theoretic complexity and coupling measures and their permutation analogues. In particular, we show the following two results within the realm of hidden Markov models with ergodic internal processes: the two permutation analogues of the transfer entropy, the symbolic transfer entropy and the transfer entropy on rank vectors, are both equivalent to the transfer entropy if they are considered as the rates, and the directed information theory can be captured by the permutation entropy approach. △ Less

Submitted 26 June, 2013; v1 submitted 9 April, 2012; originally announced April 2012.

Comments: 26 pages

Symbolic transfer entropy rate is equal to transfer entropy rate for bivariate finite-alphabet stationary ergodic Markov processes

Authors: Taichi Haruna, Kohei Nakajima

Abstract: Transfer entropy is a measure of the magnitude and the direction of information flow between jointly distributed stochastic processes. In recent years, its permutation analogues are considered in the literature to estimate the transfer entropy by counting the number of occurrences of orderings of values, not the values themselves. It has been suggested that the method of permutation is easy to imp… ▽ More Transfer entropy is a measure of the magnitude and the direction of information flow between jointly distributed stochastic processes. In recent years, its permutation analogues are considered in the literature to estimate the transfer entropy by counting the number of occurrences of orderings of values, not the values themselves. It has been suggested that the method of permutation is easy to implement, computationally low cost and robust to noise when applying to real world time series data. In this paper, we initiate a theoretical treatment of the corresponding rates. In particular, we consider the transfer entropy rate and its permutation analogue, the symbolic transfer entropy rate, and show that they are equal for any bivariate finite-alphabet stationary ergodic Markov process. This result is an illustration of the duality method introduced in [T. Haruna and K. Nakajima, Physica D 240, 1370 (2011)]. We also discuss the relationship among the transfer entropy rate, the time-delayed mutual information rate and their permutation analogues. △ Less

Submitted 11 March, 2013; v1 submitted 12 December, 2011; originally announced December 2011.

Comments: 18 pages

Permutation Excess Entropy and Mutual Information between the Past and Future

Authors: Taichi Haruna, Kohei Nakajima

Abstract: We address the excess entropy, which is a measure of complexity for stationary time series, from the ordinal point of view. We show that the permutation excess entropy is equal to the mutual information between two adjacent semi-infinite blocks in the space of orderings for finite-state stationary ergodic Markov processes. This result may shed a new light on the relationship between complexity and… ▽ More We address the excess entropy, which is a measure of complexity for stationary time series, from the ordinal point of view. We show that the permutation excess entropy is equal to the mutual information between two adjacent semi-infinite blocks in the space of orderings for finite-state stationary ergodic Markov processes. This result may shed a new light on the relationship between complexity and anticipation. △ Less

Submitted 12 December, 2011; originally announced December 2011.

Comments: 14 pages, 10th International Conference on Computing Anticipatory Systems, Liege, Belgium, August 11, 2011

Permutation Complexity via Duality between Values and Orderings

Authors: Taichi Haruna, Kohei Nakajima

Abstract: We study the permutation complexity of finite-state stationary stochastic processes based on a duality between values and orderings between values. First, we establish a duality between the set of all words of a fixed length and the set of all permutations of the same length. Second, on this basis, we give an elementary alternative proof of the equality between the permutation entropy rate and the… ▽ More We study the permutation complexity of finite-state stationary stochastic processes based on a duality between values and orderings between values. First, we establish a duality between the set of all words of a fixed length and the set of all permutations of the same length. Second, on this basis, we give an elementary alternative proof of the equality between the permutation entropy rate and the entropy rate for a finite-state stationary stochastic processes first proved in [Amigo, J.M., Kennel, M. B., Kocarev, L., 2005. Physica D 210, 77-95]. Third, we show that further information on the relationship between the structure of values and the structure of orderings for finite-state stationary stochastic processes beyond the entropy rate can be obtained from the established duality. In particular, we prove that the permutation excess entropy is equal to the excess entropy, which is a measure of global correlation present in a stationary stochastic process, for finite-state stationary ergodic Markov processes. △ Less

Submitted 12 December, 2011; originally announced December 2011.

Comments: 26 pages

Journal ref: Physica D 240 (2011) 1370-1377

Max-Plus Algebra for Complex Variables and Its Application to Discrete Fourier Transformation

Authors: Tetsu Yajima, Keisuke Nakajima, Naruyoshi Asano

Abstract: A generalization of the max-plus transformation, which is known as a method to derive cellular automata from integrable equations, is proposed for complex numbers. Operation rules for this transformation is also studied for general number of complex variables. As an application, the max-plus transformation is applied to the discrete Fourier transformation. Stretched coordinates are introduced to… ▽ More A generalization of the max-plus transformation, which is known as a method to derive cellular automata from integrable equations, is proposed for complex numbers. Operation rules for this transformation is also studied for general number of complex variables. As an application, the max-plus transformation is applied to the discrete Fourier transformation. Stretched coordinates are introduced to obtain the max-plus transformation whose imaginary part coinsides with a phase of the discrete Fourier transformation. △ Less

Submitted 26 May, 2005; originally announced May 2005.