Approximability of deep equilibria
Authors:
Samson Alva,
Eduardo Dueñez,
Jose Iovino,
Claire Walton
Abstract:
We introduce a structural framework for computations involving floating-point operations.Informed by real-valued logic, we introduce deep computations (ultracomputations) and deep iterates, formalizing the ideas of "asymptotic limit" of computations and compositional iterates, respectively.
As an application of this framework, we prove the existence of deep equilibria, which hitherto have been f…
▽ More
We introduce a structural framework for computations involving floating-point operations.Informed by real-valued logic, we introduce deep computations (ultracomputations) and deep iterates, formalizing the ideas of "asymptotic limit" of computations and compositional iterates, respectively.
As an application of this framework, we prove the existence of deep equilibria, which hitherto have been found only empirically (yielding remarkable memory savings in deep learning). Our proof of existence of deep equilibria is based on the concept of idempotent ultrafilter from combinatorics and inspired by the notion of indiscernibility from model theory.
We study and characterize deep computations (and hence deep equilibria) that are bona fide computable, i.e., uniformly approximable by a priori given computable primitive real-valued functions. Informed by model theory of real-valued structures, as well as Cp-theory from topology, we use a classical result of Grothendieck to characterize computability of deep computations in terms of continuous extendibility.
Our framework does not impose a priori uniform/global bounds on real-valued quantities; therefore, our structures yield non-compact types spaces. Such type spaces require a more nuanced topological treatment than compact ones arising in model theory of [0,1]-valued structures.
△ Less
Submitted 20 May, 2025; v1 submitted 9 September, 2024;
originally announced September 2024.
