-
A new approach to shooting methods for terminal value problems of fractional differential equations
Authors:
Kai Diethelm,
Frank Uhlig
Abstract:
For terminal value problems of fractional differential equations of order $α\in (0,1)$ that use Caputo derivatives, shooting methods are a well developed and investigated approach. Based on recently established analytic properties of such problems, we develop a new technique to select the required initial values that solves such shooting problems quickly and accurately. Numerical experiments indic…
▽ More
For terminal value problems of fractional differential equations of order $α\in (0,1)$ that use Caputo derivatives, shooting methods are a well developed and investigated approach. Based on recently established analytic properties of such problems, we develop a new technique to select the required initial values that solves such shooting problems quickly and accurately. Numerical experiments indicate that this new proportional secting technique converges very quickly and accurately to the solution. Run time measurements indicate a speedup factor of between 4 and 10 when compared to the standard bisection method.
△ Less
Submitted 10 August, 2023; v1 submitted 7 March, 2023;
originally announced March 2023.
-
The Challenges of Teaching Elementary Linear Algebra in a Modern Matrix Based Way
Authors:
Frank Uhlig
Abstract:
We assess the situation of our elementary Linear Algebra classes in the US holistically and through personal history recollections. Possible remedies for our elementary Linear Algebra's teaching problems are discussed and a change from abstract algebraic taught classes to a concrete matrix based first course is considered. The challenges of such modernization attempts for this course are laid out…
▽ More
We assess the situation of our elementary Linear Algebra classes in the US holistically and through personal history recollections. Possible remedies for our elementary Linear Algebra's teaching problems are discussed and a change from abstract algebraic taught classes to a concrete matrix based first course is considered. The challenges of such modernization attempts for this course are laid out in light of our increased after-Covid use of e-books and e-primers. We specifically address the useless and needless, but ubiquitous use of determinants, characteristic polynomials and polynomial root finding methods that are propagated in our elementary text books and are used in the majority of our elementary Linear Algebra classes for the matrix eigenvalue problem but that have no practical use whatsoever and offer no solution for finding matrix eigenvalues. This paper challenges all mathematicians as we have misinformed and miseducated our students badly for decades in elementary Linear Algebra now and urges a switch to a new, fully matrix theoretical approach that covers all classical subjects in a practical and computable way.
△ Less
Submitted 7 March, 2023;
originally announced March 2023.
-
Adapted AZNN Methods for Time-Varying and Static Matrix Problems
Authors:
Frank Uhlig
Abstract:
We present adapted Zhang Neural Networks (AZNN) in which the parameter settings for the exponential decay constant $η$ and the length of the start-up phase of basic ZNN are adapted to the problem at hand. Specifically we study experiments with AZNN for time-varying square matrix factorizations as a product of time-varying symmetric matrices and for the time-varying matrix square roots problem. Dif…
▽ More
We present adapted Zhang Neural Networks (AZNN) in which the parameter settings for the exponential decay constant $η$ and the length of the start-up phase of basic ZNN are adapted to the problem at hand. Specifically we study experiments with AZNN for time-varying square matrix factorizations as a product of time-varying symmetric matrices and for the time-varying matrix square roots problem. Differing from generally used small $η$ values and minimal start-up length phases in ZNN, we adapt the basic ZNN method to work with large or even gigantic $η$ settings and arbitrary length start-ups using Euler's low accuracy finite difference formula. These adaptations improve the speed of AZNN's convergence and lower its solution error bounds for our chosen problems significantly to near machine constant or even lower levels.
Parameter-varying AZNN also allows us to find full rank symmetrizers of static matrices reliably, for example for the Kahan and Frank matrices and for matrices with highly ill-conditioned eigenvalues and complicated Jordan structures of dimensions from $n = 2$ on up. This helps in cases where full rank static matrix symmetrizers have never been successfully computed before.
△ Less
Submitted 6 March, 2023; v1 submitted 20 September, 2022;
originally announced September 2022.
-
Iterative optimal solutions of linear matrix equations for Hyperspectral and Multispectral image fusing
Authors:
Frank Uhlig,
An-Bao Xu
Abstract:
For a linear matrix function $f$ in $X \in \R^{m\times n}$ we consider inhomogeneous linear matrix equations $f(X) = E$ for $E \neq 0$ that have or do not have solutions. For such systems we compute optimal norm constrained solutions iteratively using the Conjugate Gradient and Lanczos' methods in combination with the More-Sorensen optimizer. We build codes for ten linear matrix equations, of Sylv…
▽ More
For a linear matrix function $f$ in $X \in \R^{m\times n}$ we consider inhomogeneous linear matrix equations $f(X) = E$ for $E \neq 0$ that have or do not have solutions. For such systems we compute optimal norm constrained solutions iteratively using the Conjugate Gradient and Lanczos' methods in combination with the More-Sorensen optimizer. We build codes for ten linear matrix equations, of Sylvester, Lyapunov, Stein and structured types and their T-versions, that differ only in two five times repeated equation specific code lines. Numerical experiments with linear matrix equations are performed that illustrate universality and efficiency of our method for dense and small data matrices, as well as for sparse and certain structured input matrices. Specifically we show how to adapt our universal method for sparse inputs and for structured data such as encountered when fusing image data sets via a Sylvester equation algorithm to obtain an image of higher resolution.
△ Less
Submitted 1 August, 2021;
originally announced August 2021.
-
Zeroing Neural Networks : an Introduction to Predictive Computations for Time-varying Matrix Problems via ZNN
Authors:
Frank Uhlig
Abstract:
This paper wants to increase our understanding and computational know-how for time--varying matrix problems and Zhang Neural Networks (ZNNs). These neural networks were invented for time or single parameter--varying matrix problems around 2001 in China and almost all of their advances have been made in and most still come from its birthplace. Zhang Neural Network methods have become a backbone for…
▽ More
This paper wants to increase our understanding and computational know-how for time--varying matrix problems and Zhang Neural Networks (ZNNs). These neural networks were invented for time or single parameter--varying matrix problems around 2001 in China and almost all of their advances have been made in and most still come from its birthplace. Zhang Neural Network methods have become a backbone for solving discretized sensor driven time--varying matrix problems in real-time, in theory and in on--chip applications for robots, in control theory and other engineering applications in China. They have become the method of choice for many time--varying matrix problems that benefit from or require efficient, accurate and predictive real--time computations. A typical discretized Zhang Neural Network algorithm needs seven distinct steps in its initial set-up. The construction of discretized Zhang Neural Network algorithms starts from a model equation with its associated error equation and the stipulation that the error function decrease exponentially fast. The error function differential equation is then mated with a convergent look-ahead finite difference formula to create a distinctly new multi--step style solver that predicts the future state of the system reliably from current and earlier state and solution data. Matlab codes of discretized Zhang Neural Network algorithms for time varying matrix problems typically consist of one linear equations solve and one recursion of already available data per time step. This makes discretized Zhang Neural network based algorithms highly competitive with ordinary differential equation initial value analytic continuation methods for function given data that are designed to work adaptively. .
△ Less
Submitted 4 December, 2023; v1 submitted 6 August, 2020;
originally announced August 2020.
-
The Eight Epochs of Math as regards past and future Matrix Computation
Authors:
Frank Uhlig
Abstract:
This paper gives a personal assessment of Epoch making advances in Matrix Computations from antiquity and with an eye towards tomorrow.
We trace the development of number systems and elementary algebra, and the uses of Gaussian Elimination methods from around 2000 BC on to current real-time Neural Network computations to solve time-varying linear equations.
We include relevant advances from Ch…
▽ More
This paper gives a personal assessment of Epoch making advances in Matrix Computations from antiquity and with an eye towards tomorrow.
We trace the development of number systems and elementary algebra, and the uses of Gaussian Elimination methods from around 2000 BC on to current real-time Neural Network computations to solve time-varying linear equations.
We include relevant advances from China from the 3rd century AD on, and from India and Persia in the 9th century and discuss the conceptual genesis of vectors and matrices in central Europe and Japan in the 14th through 17th centuries AD.
Followed by the 150 year cul-de-sac of polynomial root finder research for matrix eigenvalues, as well as the superbly useful matrix iterative methods and Francis' eigenvalue Algorithm from last century.
Then we explain the recent use of initial value problem solvers to master time-varying linear and nonlinear matrix equations via Neural Networks.
We end with a short outlook upon new hardware schemes with multilevel processors that go beyond the 0-1 base 2 framework which all of our past and current electronic computers have been using.
△ Less
Submitted 4 August, 2020;
originally announced August 2020.
-
Constructing the Field of Values of Decomposable and General Matrices Using the ZNN Based Path Following Method
Authors:
Frank Uhlig
Abstract:
This paper describes and develops a fast and accurate path following algorithm that computes the field of values boundary curve for every conceivable complex or real square matrix $A$. It relies on a matrix flow decomposition method that finds a proper block-diagonal flow representation for the associated hermitean matrix flow ${\cal F}_A(t) = \cos(t) H + \sin(t) K$. Here ${\cal F}_A(t)$ is a 1-pa…
▽ More
This paper describes and develops a fast and accurate path following algorithm that computes the field of values boundary curve for every conceivable complex or real square matrix $A$. It relies on a matrix flow decomposition method that finds a proper block-diagonal flow representation for the associated hermitean matrix flow ${\cal F}_A(t) = \cos(t) H + \sin(t) K$. Here ${\cal F}_A(t)$ is a 1-parameter-varying linear combination of the real and skew part matrices $H = (A+A^*)/2$ and $K = (A-A^*)/(2i)$ of $A$. For decomposing flows ${\cal F}_A(t)$, the algorithm decomposes a given dense general matrix $A$ unitarily into block-diagonal form $U^*AU = \text { diag} (A_j)$ with $j > 1$ diagonal blocks $A_j$ whose individual sizes add up to the size of $A$. It then computes the field of values boundaries separately for each diagonal block $A_j$ using the path following ZNN eigenvalue method. The convex hull of all sub-fields of values boundary points then determines the field of values boundary curve correctly for decomposing and non-decomposing matrices $A$. The algorithm removes standard restrictions for path following FoV methods that generally cannot deal with decomposing matrices $A$ due to possible eigencurve crossings of ${\cal F}_A(t)$. Tests and numerical comparisons are included. Our ZNN based method is coded for sequential and parallel computations and both versions run very accurately and very fast when compared with Johnson's Francis QR eigenvalue and Bendixon rectangle based method that computes complete eigenanalyses of ${\cal F}_A(t_k)$ for every chosen $t_k \in {[} 0,2π{]}$ more slowly.
△ Less
Submitted 11 August, 2022; v1 submitted 1 June, 2020;
originally announced June 2020.
-
On the Block-Decomposability of 1-Parameter Matrix Flows and Static Matrices
Authors:
Frank Uhlig
Abstract:
For general complex or real 1-parameter matrix flows $A(t)_{n,n}$ and for time-invariant static matrices $A \in \CC_{n,n}$ alike, this paper considers ways to decompose matrix flows and single matrices globally via one constant matrix similarity $C_{n,n}$ as $A(t) = C ^{-1} \cdot \text{ diag}(A_1(t), ..., A_\ell(t)) \cdot C$ or $A = C^{-1}\cdot \text{diag}(A_1,...,A_\ell)\cdot C$ with each diagona…
▽ More
For general complex or real 1-parameter matrix flows $A(t)_{n,n}$ and for time-invariant static matrices $A \in \CC_{n,n}$ alike, this paper considers ways to decompose matrix flows and single matrices globally via one constant matrix similarity $C_{n,n}$ as $A(t) = C ^{-1} \cdot \text{ diag}(A_1(t), ..., A_\ell(t)) \cdot C$ or $A = C^{-1}\cdot \text{diag}(A_1,...,A_\ell)\cdot C$ with each diagonal block $A_k(t)$ or $A_k$ square and their number $\ell > 1$ if this is possible. The theory behind our proposed algorithm is elementary and uses the concept of invariant subspaces for the Matlab {\tt eig} computed 'eigenvectors' of one associated flow matrix $B(t_a)$ to find the coarsest simultaneous block structure for all flow matrices $B(t_b)$. The method works very efficiently for all time-varying matrix flows, be they differentiable, continuous or discontinuous in $t$, and for all fixed entry matrices $A$; as well as for all types of square matrix flows or fixed entry matrices such as hermitean, real symmetric, normal or general complex and real flows $A(t)$ or static matrices $A$, with or without Jordan block structures and with or without repeated eigenvalues. Our intended aim is to discover diagonal-block decomposable flows as they originate in sensor driven outputs for time-varying matrix problems and thereby help to reduce the complexities of their numerical treatments through adapting 'divide and conquer' methods for their diagonal sub-blocks. Our method is also applicable to standard fixed entry matrices of all structures and types. In the process we discover and study k-normal fixed entry matrix classes that can be decomposed under unitary similarities into various $k$-dimensional block-diagonal forms.
△ Less
Submitted 19 November, 2020; v1 submitted 1 June, 2020;
originally announced June 2020.
-
Coalescing Eigenvalues and Crossing Eigencurves of 1-Parameter Matrix Flows
Authors:
Frank Uhlig
Abstract:
We investigate the eigenvalue curves of 1-parameter hermitean and general complex or real matrix flows $A(t)$ in light of their geometry and the uniform decomposability of $A(t)$ for all parameters $t$. The often misquoted and misapplied results by Hund and von Neumann and by Wigner for eigencurve crossings from the late 1920s are clarified for hermitean matrix flows $A(t) = (A(t))^*$. A conjectur…
▽ More
We investigate the eigenvalue curves of 1-parameter hermitean and general complex or real matrix flows $A(t)$ in light of their geometry and the uniform decomposability of $A(t)$ for all parameters $t$. The often misquoted and misapplied results by Hund and von Neumann and by Wigner for eigencurve crossings from the late 1920s are clarified for hermitean matrix flows $A(t) = (A(t))^*$. A conjecture on extending these results to general non-normal or non-hermitean 1-parameter matrix flows is formulated and investigated. An algorithm to compute the block dimensions of uniformly decomposable hermitean matrix flows is described and tested. The algorithm uses the ZNN method to compute the time-varying matrix eigenvalue curves of $A(t)$ for $t_o \leq t\leq t_f$. Similar efforts for general complex matrix flows are described. This extension leads to many new and open problems. Specifically, we point to the difficult relationship between the geometry of eigencurves for general complex matrix flows $A(t)$ and a general flow's decomposability into blockdiagonal form via one fixed unitary or general matrix similarity for all parameters $t$.
△ Less
Submitted 4 February, 2020;
originally announced February 2020.
-
The Construction of High Order Convergent Look-Ahead Finite Difference Formulas for Zhang Neural Networks
Authors:
Frank Uhlig
Abstract:
Zhang Neural Networks rely on convergent 1-step ahead finite difference formulas of which very few are known. Those which are known have been constructed in ad-hoc ways and suffer from low truncation error orders. This paper develops a constructive method to find convergent look-ahead finite difference schemes of higher truncation error orders. The method consists of seeding the free variables of…
▽ More
Zhang Neural Networks rely on convergent 1-step ahead finite difference formulas of which very few are known. Those which are known have been constructed in ad-hoc ways and suffer from low truncation error orders. This paper develops a constructive method to find convergent look-ahead finite difference schemes of higher truncation error orders. The method consists of seeding the free variables of a linear system comprised of Taylor expansion coefficients followed by a minimization algorithm for the maximal magnitude root of the formula's characteristic polynomial. This helps us find new convergent 1-step ahead finite difference formulas of any truncation error order. Once a polynomial has been found with roots inside the complex unit circle and no repeated roots on it, the associated look-ahead ZNN discretization formula is convergent and can be used for solving any discretized ZNN based model. Our method recreates and validates the few known convergent formulas, all of which have truncation error orders at most 4. It also creates new convergent 1-step ahead difference formulas with truncation error orders 5 through 8.
△ Less
Submitted 23 April, 2019;
originally announced April 2019.
-
Zhang Neural Networks for Fast and Accurate Computations of the Field of Values
Authors:
Frank Uhlig
Abstract:
In this paper a new and different neural network, called Zhang Neural Network (ZNN) is appropriated from discrete time-varying matrix problems and applied to the angle parameter-varying matrix field of values (FoV) problem. This problem acts as a test bed for newly discovered convergent 1-step ahead finite difference formulas of high truncation orders. The ZNN method that uses a look-ahead finite…
▽ More
In this paper a new and different neural network, called Zhang Neural Network (ZNN) is appropriated from discrete time-varying matrix problems and applied to the angle parameter-varying matrix field of values (FoV) problem. This problem acts as a test bed for newly discovered convergent 1-step ahead finite difference formulas of high truncation orders. The ZNN method that uses a look-ahead finite difference scheme of error order 6 gives us 15+ accurate digits of the FoV boundary in record time when applied to hermitean matrix flows $A(t)$.
△ Less
Submitted 23 April, 2019;
originally announced April 2019.
-
Time-Varying Matrix Eigenanalyses via Zhang Neural Networks and look-Ahead Finite Difference Equations
Authors:
Frank Uhlig,
Yunong Zhang
Abstract:
This paper adapts look-ahead and backward finite difference formulas to compute future eigenvectors and eigenvalues of piecewise smooth time-varying symmetric matrix flows $A(t)$. It is based on the Zhang Neural Network (ZNN) model for time-varying problems and uses the associated error function $E(t) = A(t)V(t) - V(t) D(t)$ or $e_i(t) = A(t)v_i(t) -\la_i(t)v_i(t)$ with the Zhang design stipulatio…
▽ More
This paper adapts look-ahead and backward finite difference formulas to compute future eigenvectors and eigenvalues of piecewise smooth time-varying symmetric matrix flows $A(t)$. It is based on the Zhang Neural Network (ZNN) model for time-varying problems and uses the associated error function $E(t) = A(t)V(t) - V(t) D(t)$ or $e_i(t) = A(t)v_i(t) -\la_i(t)v_i(t)$ with the Zhang design stipulation that $\dot E(t) = - ηE(t)$ or $\dot e_i(t) = - ηe_i(t)$ with $η> 0$ so that $E(t)$ and $e(t)$ decrease exponentially over time. This leads to a discrete-time differential equation of the form $P(t_k) \dot z(t_k) = q(t_k)$ for the eigendata vector $z(t_k)$ of $A(t_k)$. Convergent look-ahead finite difference formulas of varying error orders then allow us to express $z(t_{k+1})$ in terms of earlier $A$ and $z$ data. Numerical tests, comparisons and open questions complete the paper.
△ Less
Submitted 23 April, 2019;
originally announced April 2019.
