-
Construction of the Kolmogorov-Arnold representation using the Newton-Kaczmarz method
Authors:
Michael Poluektov,
Andrew Polar
Abstract:
It is known that any continuous multivariate function can be represented exactly by a composition functions of a single variable - the so-called Kolmogorov-Arnold representation. It can be a convenient tool for tasks where it is required to obtain a predictive model that maps some vector input of a black box system into a scalar output. In this case, the representation may not be exact, and it is…
▽ More
It is known that any continuous multivariate function can be represented exactly by a composition functions of a single variable - the so-called Kolmogorov-Arnold representation. It can be a convenient tool for tasks where it is required to obtain a predictive model that maps some vector input of a black box system into a scalar output. In this case, the representation may not be exact, and it is more correct to refer to such structure as the Kolmogorov-Arnold model (or, as more recently popularised, 'network'). Construction of such model based on the recorded input-output data is a challenging task. In the present paper, it is suggested to decompose the underlying functions of the representation into continuous basis functions and parameters. It is then proposed to find the parameters using the Newton-Kaczmarz method for solving systems of non-linear equations. The algorithm is then modified to support parallelisation. The paper demonstrates that such approach is also an excellent tool for data-driven solution of partial differential equations. Numerical examples show that for the considered model, the Newton-Kaczmarz method for parameter estimation is efficient and more robust with respect to the section of the initial guess than the straightforward application of the Gauss-Newton method. Finally, the Kolmogorov-Arnold model is compared to the MATLAB's built-in neural networks on a relatively large-scale problem (25 inputs, datasets of 10 million records), significantly outperforming the multilayer perceptrons (MLPs) in this particular problem (4-10 minutes vs. 4-8 hours of training time, as well as higher accuracy, lower CPU usage, and smaller memory footprint).
△ Less
Submitted 31 January, 2025; v1 submitted 14 May, 2023;
originally announced May 2023.
-
A deep machine learning algorithm for construction of the Kolmogorov-Arnold representation
Authors:
Andrew Polar,
Michael Poluektov
Abstract:
The Kolmogorov-Arnold representation is a proven adequate replacement of a continuous multivariate function by an hierarchical structure of multiple functions of one variable. The proven existence of such representation inspired many researchers to search for a practical way of its construction, since such model answers the needs of machine learning. This article shows that the Kolmogorov-Arnold r…
▽ More
The Kolmogorov-Arnold representation is a proven adequate replacement of a continuous multivariate function by an hierarchical structure of multiple functions of one variable. The proven existence of such representation inspired many researchers to search for a practical way of its construction, since such model answers the needs of machine learning. This article shows that the Kolmogorov-Arnold representation is not only a composition of functions but also a particular case of a tree of the discrete Urysohn operators. The article introduces new, quick and computationally stable algorithm for constructing of such Urysohn trees. Besides continuous multivariate functions, the suggested algorithm covers the cases with quantised inputs and combination of quantised and continuous inputs. The article also contains multiple results of testing of the suggested algorithm on publicly available datasets, used also by other researchers for benchmarking.
△ Less
Submitted 22 June, 2020; v1 submitted 14 January, 2020;
originally announced January 2020.
-
Canonical block-oriented model
Authors:
Andrew Polar,
Michael Poluektov
Abstract:
The block-oriented models are usually based on linear dynamic and non-linear static blocks that are connected in various sequential/parallel ways. Some particular configurations of the involved blocks result in the well-known Hammerstein, Wiener, Hammerstein-Wiener and generalised Hammerstein models. The Urysohn model is a lesser-known model; it is represented by a single non-linear dynamic block…
▽ More
The block-oriented models are usually based on linear dynamic and non-linear static blocks that are connected in various sequential/parallel ways. Some particular configurations of the involved blocks result in the well-known Hammerstein, Wiener, Hammerstein-Wiener and generalised Hammerstein models. The Urysohn model is a lesser-known model; it is represented by a single non-linear dynamic block and can be approximated by a number of parallel Hammerstein blocks. In this paper, it is shown that any block-oriented model can be adequately replaced by a single Urysohn block followed by a single static non-linear block. Furthermore, a method of the so-called non-parametric identification of such object is introduced.
△ Less
Submitted 29 September, 2019;
originally announced September 2019.
-
Modelling non-linear control systems using the discrete Urysohn operator
Authors:
Michael Poluektov,
Andrew Polar
Abstract:
This paper introduces a multiple-input discrete Urysohn operator for modelling non-linear control systems and a technique of its identification by processing the observed input and output signals. It is shown that, due to the nature of the discrete Urysohn operator, the identification problem always has an infinity of solutions, which exactly convert the inputs to the output. The suggested iterati…
▽ More
This paper introduces a multiple-input discrete Urysohn operator for modelling non-linear control systems and a technique of its identification by processing the observed input and output signals. It is shown that, due to the nature of the discrete Urysohn operator, the identification problem always has an infinity of solutions, which exactly convert the inputs to the output. The suggested iterative identification procedure, however, leads to a unique solution with the minimum norm, requires only few arithmetic operations with the parameter values and is applicable to a real-time identification, running concurrently with the data reading. The efficiency of the proposed modelling and identification approaches is demonstrated using an example of a non-linear mechanical system, which is represented by a differential equation, and an example of a complex real-world dynamic object.
△ Less
Submitted 4 April, 2020; v1 submitted 5 February, 2018;
originally announced February 2018.
-
Atomistic-continuum multiscale modelling of magnetisation dynamics at non-zero temperature
Authors:
Doghonay Arjmand,
Mikhail Poluektov,
Gunilla Kreiss
Abstract:
In this article, a few problems related to multiscale modelling of magnetic materials at finite temperatures and possible ways of solving these problems are discussed. The discussion is mainly centred around two established multiscale concepts: the partitioned domain and the upscaling-based methodologies. The major challenge for both multiscale methods is to capture the correct value of magnetisat…
▽ More
In this article, a few problems related to multiscale modelling of magnetic materials at finite temperatures and possible ways of solving these problems are discussed. The discussion is mainly centred around two established multiscale concepts: the partitioned domain and the upscaling-based methodologies. The major challenge for both multiscale methods is to capture the correct value of magnetisation length accurately, which is affected by a random temperature-dependent force. Moreover, general limitations of these multiscale techniques in application to spin systems are discussed.
△ Less
Submitted 16 February, 2017;
originally announced February 2017.
