-
Operator Learning Meets Numerical Analysis: Improving Neural Networks through Iterative Methods
Authors:
Emanuele Zappala,
Daniel Levine,
Sizhuang He,
Syed Rizvi,
Sacha Levy,
David van Dijk
Abstract:
Deep neural networks, despite their success in numerous applications, often function without established theoretical foundations. In this paper, we bridge this gap by drawing parallels between deep learning and classical numerical analysis. By framing neural networks as operators with fixed points representing desired solutions, we develop a theoretical framework grounded in iterative methods for…
▽ More
Deep neural networks, despite their success in numerous applications, often function without established theoretical foundations. In this paper, we bridge this gap by drawing parallels between deep learning and classical numerical analysis. By framing neural networks as operators with fixed points representing desired solutions, we develop a theoretical framework grounded in iterative methods for operator equations. Under defined conditions, we present convergence proofs based on fixed point theory. We demonstrate that popular architectures, such as diffusion models and AlphaFold, inherently employ iterative operator learning. Empirical assessments highlight that performing iterations through network operators improves performance. We also introduce an iterative graph neural network, PIGN, that further demonstrates benefits of iterations. Our work aims to enhance the understanding of deep learning by merging insights from numerical analysis, potentially guiding the design of future networks with clearer theoretical underpinnings and improved performance.
△ Less
Submitted 2 October, 2023;
originally announced October 2023.
-
When mutually subisomorphic Baer modules are isomorphic
Authors:
Najmeh Dehghani,
S. Tariq Rizvi
Abstract:
The Schröder-Bernstein Theorem for sets is well known. The question of whether two subisomorphic algebraic structures are isomorphic to each other, is of interest. An $R$-module $M$ is said to satisfy the Schröder-Bernstein (or SB) property if any pair of direct summands of $M$ are isomorphic provided that each one is isomorphic to a direct summand of the other. A ring $R$ (with an involution…
▽ More
The Schröder-Bernstein Theorem for sets is well known. The question of whether two subisomorphic algebraic structures are isomorphic to each other, is of interest. An $R$-module $M$ is said to satisfy the Schröder-Bernstein (or SB) property if any pair of direct summands of $M$ are isomorphic provided that each one is isomorphic to a direct summand of the other. A ring $R$ (with an involution $\star$) is called a Baer (Baer $\star$-)ring if the right annihilator of every nonempty subset of $R$ is generated by an idempotent (a projection). It is clear that every Baer $\star$-ring is a Baer ring. Kaplansky showed that Baer $\star$-rings satisfy the SB property. This motivated us to investigate whether any Baer ring satisfies the SB property. In this paper we carry out a study of this question and investigate when two subisomorphic Baer modules are isomorphic. Besides, we study extending modules which satisfy the SB property. We characterize a commutative domain $R$ over which any pair of subisomorphic extending modules are isomorphic.
△ Less
Submitted 8 September, 2019;
originally announced September 2019.
-
General Viscosity Implicit Midpoint Rule For Nonexpansive Mapping
Authors:
Shuja Haider Rizvi
Abstract:
In this work, we suggest a general viscosity implicit midpoint rule for nonexpansive mapping in the framework of Hilbert space. Further, under the certain conditions imposed on the sequence of parameters, strong convergence theorem is proved by the sequence generated by the proposed iterative scheme, which, in addition, is the unique solution of the variational inequality problem. Furthermore, we…
▽ More
In this work, we suggest a general viscosity implicit midpoint rule for nonexpansive mapping in the framework of Hilbert space. Further, under the certain conditions imposed on the sequence of parameters, strong convergence theorem is proved by the sequence generated by the proposed iterative scheme, which, in addition, is the unique solution of the variational inequality problem. Furthermore, we provide some applications to variational inequalities, Fredholm integral equations, and nonlinear evolution equations. The results presented in this work may be treated as an improvement, extension and refinement of some corresponding ones in the literature.
△ Less
Submitted 20 September, 2016;
originally announced September 2016.
-
On cycle-supermagic labelings of the disconnected graphs
Authors:
Syed Tahir Raza Rizvi,
Kashif Ali
Abstract:
In this paper we formulate cycle-supermagic labelings for the disjoint union of isomorphic copies of different families of graphs. We also prove that disjoint union of non isomorphic copies of fans and ladders are cycle-supermagic.
In this paper we formulate cycle-supermagic labelings for the disjoint union of isomorphic copies of different families of graphs. We also prove that disjoint union of non isomorphic copies of fans and ladders are cycle-supermagic.
△ Less
Submitted 23 April, 2015;
originally announced June 2015.
