Associative Memory and Generative Diffusion in the Zero-noise Limit
Authors:
Joshua Hess,
Quaid Morris
Abstract:
Connections between generative diffusion and continuous-state associative memory models are studied. Morse-Smale dynamical systems are emphasized as universal approximators of gradient-based associative memory models and diffusion models as white-noise perturbed systems thereof. Universal properties of associative memory that follow from this description are described and used to characterize a ge…
▽ More
Connections between generative diffusion and continuous-state associative memory models are studied. Morse-Smale dynamical systems are emphasized as universal approximators of gradient-based associative memory models and diffusion models as white-noise perturbed systems thereof. Universal properties of associative memory that follow from this description are described and used to characterize a generic transition from generation to memory as noise levels diminish. Structural stability inherited by Morse-Smale flows is shown to imply a notion of stability for diffusions at vanishing noise levels. Applied to one- and two-parameter families of gradients, this indicates stability at all but isolated points of associative memory learning landscapes and the learning and generation landscapes of diffusion models with gradient drift in the zero-noise limit, at which small sets of generic bifurcations characterize qualitative transitions between stable systems. Examples illustrating the characterization of these landscapes by sequences of these bifurcations are given, along with structural stability criterion for classic and modern Hopfield networks (equivalently, the attention mechanism).
△ Less
Submitted 5 June, 2025;
originally announced June 2025.
A Solution Set-Based Entropy Principle for Constitutive Modeling in Mechanics
Authors:
J. Heß,
A. F. Cheviakov
Abstract:
Entropy principles based on thermodynamic consistency requirements are widely used for constitutive modeling in continuum mechanics, providing physical constraints on a priori unknown constitutive functions. The well-known Müller-Liu procedure is based on Liu's lemma for linear systems. While the Müller-Liu algorithm works well for basic models with simple constitutive dependencies, it cannot take…
▽ More
Entropy principles based on thermodynamic consistency requirements are widely used for constitutive modeling in continuum mechanics, providing physical constraints on a priori unknown constitutive functions. The well-known Müller-Liu procedure is based on Liu's lemma for linear systems. While the Müller-Liu algorithm works well for basic models with simple constitutive dependencies, it cannot take into account nonlinear relationships that exist between higher derivatives of the fields in the cases of more complex constitutive dependencies.
The current contribution presents a general solution set-based procedure, which, for a model system of differential equations, respects the geometry of the solution manifold, and yields a set of constraint equations on the unknown constitutive functions, which are necessary and sufficient conditions for the entropy production to stay nonnegative for any solution. Similarly to the Müller-Liu procedure, the solution set approach is algorithmic, its output being a set of constraint equations and a residual entropy inequality. The solution set method is applicable to virtually any physical model, allows for arbitrary initially postulated forms of the constitutive dependencies, and does not use artificial constructs like Lagrange multipliers. A Maple implementation makes the solution set method computationally straightforward and useful for the constitutive modeling of complex systems.
Several computational examples are considered, in particular, models of gas, anisotropic fluid, and granular flow dynamics. The resulting constitutive function forms are analyzed, and comparisons are provided. It is shown how the solution set entropy principle can yield classification problems, leading to several complementary sets of admissible constitutive functions; such problems have not previously appeared in the constitutive modeling literature.
△ Less
Submitted 12 November, 2018;
originally announced November 2018.
