-
Understanding In-Context Learning on Structured Manifolds: Bridging Attention to Kernel Methods
Authors:
Zhaiming Shen,
Alexander Hsu,
Rongjie Lai,
Wenjing Liao
Abstract:
While in-context learning (ICL) has achieved remarkable success in natural language and vision domains, its theoretical understanding--particularly in the context of structured geometric data--remains unexplored. In this work, we initiate a theoretical study of ICL for regression of Hölder functions on manifolds. By establishing a novel connection between the attention mechanism and classical kern…
▽ More
While in-context learning (ICL) has achieved remarkable success in natural language and vision domains, its theoretical understanding--particularly in the context of structured geometric data--remains unexplored. In this work, we initiate a theoretical study of ICL for regression of Hölder functions on manifolds. By establishing a novel connection between the attention mechanism and classical kernel methods, we derive generalization error bounds in terms of the prompt length and the number of training tasks. When a sufficient number of training tasks are observed, transformers give rise to the minimax regression rate of Hölder functions on manifolds, which scales exponentially with the intrinsic dimension of the manifold, rather than the ambient space dimension. Our result also characterizes how the generalization error scales with the number of training tasks, shedding light on the complexity of transformers as in-context algorithm learners. Our findings provide foundational insights into the role of geometry in ICL and novels tools to study ICL of nonlinear models.
△ Less
Submitted 12 June, 2025;
originally announced June 2025.
-
Transformers for Learning on Noisy and Task-Level Manifolds: Approximation and Generalization Insights
Authors:
Zhaiming Shen,
Alex Havrilla,
Rongjie Lai,
Alexander Cloninger,
Wenjing Liao
Abstract:
Transformers serve as the foundational architecture for large language and video generation models, such as GPT, BERT, SORA and their successors. Empirical studies have demonstrated that real-world data and learning tasks exhibit low-dimensional structures, along with some noise or measurement error. The performance of transformers tends to depend on the intrinsic dimension of the data/tasks, thou…
▽ More
Transformers serve as the foundational architecture for large language and video generation models, such as GPT, BERT, SORA and their successors. Empirical studies have demonstrated that real-world data and learning tasks exhibit low-dimensional structures, along with some noise or measurement error. The performance of transformers tends to depend on the intrinsic dimension of the data/tasks, though theoretical understandings remain largely unexplored for transformers. This work establishes a theoretical foundation by analyzing the performance of transformers for regression tasks involving noisy input data on a manifold. Specifically, the input data are in a tubular neighborhood of a manifold, while the ground truth function depends on the projection of the noisy data onto the manifold. We prove approximation and generalization errors which crucially depend on the intrinsic dimension of the manifold. Our results demonstrate that transformers can leverage low-complexity structures in learning task even when the input data are perturbed by high-dimensional noise. Our novel proof technique constructs representations of basic arithmetic operations by transformers, which may hold independent interest.
△ Less
Submitted 13 June, 2025; v1 submitted 6 May, 2025;
originally announced May 2025.
-
Partial data inverse problems for the nonlinear magnetic Schrödinger equation
Authors:
Ru-Yu Lai,
Gunther Uhlmann,
Lili Yan
Abstract:
In this paper, we study the partial data inverse problem for nonlinear magnetic Schrödinger equations. We show that the knowledge of the Dirichlet-to-Neumann map, measured on an arbitrary part of the boundary, determines the time-dependent linear coefficients, electric and magnetic potentials, and nonlinear coefficients, provided that the divergence of the magnetic potential is given. Additionally…
▽ More
In this paper, we study the partial data inverse problem for nonlinear magnetic Schrödinger equations. We show that the knowledge of the Dirichlet-to-Neumann map, measured on an arbitrary part of the boundary, determines the time-dependent linear coefficients, electric and magnetic potentials, and nonlinear coefficients, provided that the divergence of the magnetic potential is given. Additionally, we also investigate both the forward and inverse problems for the linear magnetic Schrödinger equation with a time-dependent leading term. In particular, all coefficients are uniquely recovered from boundary data.
△ Less
Submitted 10 November, 2024;
originally announced November 2024.
-
Inverse problems for time-dependent nonlinear transport equations
Authors:
Ru-Yu Lai,
Hanming Zhou
Abstract:
In this work, we investigate inverse problems of recovering the time-dependent coefficient in the nonlinear transport equation in both cases: two-dimensional Riemannian manifolds and Euclidean space $\mathbb{R}^n$, $n\geq 2$. Specifically, it is shown that its initial boundary value problem is well-posed for small initial and incoming data. Moreover, the time-dependent coefficient appearing in the…
▽ More
In this work, we investigate inverse problems of recovering the time-dependent coefficient in the nonlinear transport equation in both cases: two-dimensional Riemannian manifolds and Euclidean space $\mathbb{R}^n$, $n\geq 2$. Specifically, it is shown that its initial boundary value problem is well-posed for small initial and incoming data. Moreover, the time-dependent coefficient appearing in the nonlinear term can be uniquely determined from boundary measurements as well as initial and final data. To achieve this, the central techniques we utilize include the linearization technique and the construction of special geometrical optics solutions for the linear transport equation. This allows us to reduce the inverse coefficient problem to the inversion of certain weighted light ray transforms. Based on the developed methodology, the inverse source problem for the nonlinear transport equation in the scattering-free media is also studied.
△ Less
Submitted 30 September, 2024;
originally announced October 2024.
-
Unsupervised Solution Operator Learning for Mean-Field Games via Sampling-Invariant Parametrizations
Authors:
Han Huang,
Rongjie Lai
Abstract:
Recent advances in deep learning has witnessed many innovative frameworks that solve high dimensional mean-field games (MFG) accurately and efficiently. These methods, however, are restricted to solving single-instance MFG and demands extensive computational time per instance, limiting practicality. To overcome this, we develop a novel framework to learn the MFG solution operator. Our model takes…
▽ More
Recent advances in deep learning has witnessed many innovative frameworks that solve high dimensional mean-field games (MFG) accurately and efficiently. These methods, however, are restricted to solving single-instance MFG and demands extensive computational time per instance, limiting practicality. To overcome this, we develop a novel framework to learn the MFG solution operator. Our model takes a MFG instances as input and output their solutions with one forward pass. To ensure the proposed parametrization is well-suited for operator learning, we introduce and prove the notion of sampling invariance for our model, establishing its convergence to a continuous operator in the sampling limit. Our method features two key advantages. First, it is discretization-free, making it particularly suitable for learning operators of high-dimensional MFGs. Secondly, it can be trained without the need for access to supervised labels, significantly reducing the computational overhead associated with creating training datasets in existing operator learning methods. We test our framework on synthetic and realistic datasets with varying complexity and dimensionality to substantiate its robustness.
△ Less
Submitted 23 April, 2024; v1 submitted 27 January, 2024;
originally announced January 2024.
-
A Bilevel Optimization Method for Inverse Mean-Field Games
Authors:
Jiajia Yu,
Quan Xiao,
Tianyi Chen,
Rongjie Lai
Abstract:
In this paper, we introduce a bilevel optimization framework for addressing inverse mean-field games, alongside an exploration of numerical methods tailored for this bilevel problem. The primary benefit of our bilevel formulation lies in maintaining the convexity of the objective function and the linearity of constraints in the forward problem. Our paper focuses on inverse mean-field games charact…
▽ More
In this paper, we introduce a bilevel optimization framework for addressing inverse mean-field games, alongside an exploration of numerical methods tailored for this bilevel problem. The primary benefit of our bilevel formulation lies in maintaining the convexity of the objective function and the linearity of constraints in the forward problem. Our paper focuses on inverse mean-field games characterized by unknown obstacles and metrics. We show numerical stability for these two types of inverse problems. More importantly, we, for the first time, establish the identifiability of the inverse mean-field game with unknown obstacles via the solution of the resultant bilevel problem. The bilevel approach enables us to employ an alternating gradient-based optimization algorithm with a provable convergence guarantee. To validate the effectiveness of our methods in solving the inverse problems, we have designed comprehensive numerical experiments, providing empirical evidence of its efficacy.
△ Less
Submitted 10 January, 2024;
originally announced January 2024.
-
Reconstruction of the Doping Profile in Vlasov-Poisson
Authors:
Ru-Yu Lai,
Qin Li,
Weiran Sun
Abstract:
We study the inverse problem of recovering the doping profile in the stationary Vlasov-Poisson equation, given the knowledge of the incoming and outgoing measurements at the boundary of the domain. This problem arises from identifying impurities in the semiconductor manufacturing. Our result states that, under suitable assumptions, the doping profile can be uniquely determined through an asymptoti…
▽ More
We study the inverse problem of recovering the doping profile in the stationary Vlasov-Poisson equation, given the knowledge of the incoming and outgoing measurements at the boundary of the domain. This problem arises from identifying impurities in the semiconductor manufacturing. Our result states that, under suitable assumptions, the doping profile can be uniquely determined through an asymptotic formula of the electric field that it generates.
△ Less
Submitted 9 January, 2024;
originally announced January 2024.
-
Stable determination of time-dependent collision kernel in the nonlinear Boltzmann equation
Authors:
Ru-Yu Lai,
Lili Yan
Abstract:
We consider an inverse problem for the nonlinear Boltzmann equation with a time-dependent kernel in dimensions $n\ge 2$. We establish a logarithm-type stability result for the collision kernel from measurements under certain additional conditions. A uniqueness result is derived as an immediate consequence of the stability result. Our approach relies on second-order linearization, multivariate fini…
▽ More
We consider an inverse problem for the nonlinear Boltzmann equation with a time-dependent kernel in dimensions $n\ge 2$. We establish a logarithm-type stability result for the collision kernel from measurements under certain additional conditions. A uniqueness result is derived as an immediate consequence of the stability result. Our approach relies on second-order linearization, multivariate finite differences, as well as the stability of the light-ray transform.
△ Less
Submitted 6 September, 2023;
originally announced September 2023.
-
Partial Data Inverse Problems for the Nonlinear Schrödinger Equation
Authors:
Ru-Yu Lai,
Xuezhu Lu,
Ting Zhou
Abstract:
In this paper we prove the uniqueness and stability in determining a time-dependent nonlinear coefficient $β(t, x)$ in the Schrödinger equation $(i\partial_t + Δ+ q(t, x))u + βu^2 = 0$, from the boundary Dirichlet-to-Neumann (DN) map. In particular, we are interested in the partial data problem, in which the DN-map is measured on a proper subset of the boundary. We show two results: a local unique…
▽ More
In this paper we prove the uniqueness and stability in determining a time-dependent nonlinear coefficient $β(t, x)$ in the Schrödinger equation $(i\partial_t + Δ+ q(t, x))u + βu^2 = 0$, from the boundary Dirichlet-to-Neumann (DN) map. In particular, we are interested in the partial data problem, in which the DN-map is measured on a proper subset of the boundary. We show two results: a local uniqueness of the coefficient at the points where certain type of geometric optics (GO) solutions can reach; and a stability estimate based on the unique continuation property for the linear equation.
△ Less
Submitted 6 November, 2023; v1 submitted 28 June, 2023;
originally announced June 2023.
-
Semi-Supervised Manifold Learning with Complexity Decoupled Chart Autoencoders
Authors:
Stefan C. Schonsheck,
Scott Mahan,
Timo Klock,
Alexander Cloninger,
Rongjie Lai
Abstract:
Autoencoding is a popular method in representation learning. Conventional autoencoders employ symmetric encoding-decoding procedures and a simple Euclidean latent space to detect hidden low-dimensional structures in an unsupervised way. Some modern approaches to novel data generation such as generative adversarial networks askew this symmetry, but still employ a pair of massive networks--one to ge…
▽ More
Autoencoding is a popular method in representation learning. Conventional autoencoders employ symmetric encoding-decoding procedures and a simple Euclidean latent space to detect hidden low-dimensional structures in an unsupervised way. Some modern approaches to novel data generation such as generative adversarial networks askew this symmetry, but still employ a pair of massive networks--one to generate the image and another to judge the images quality based on priors learned from a training set. This work introduces a chart autoencoder with an asymmetric encoding-decoding process that can incorporate additional semi-supervised information such as class labels. Besides enhancing the capability for handling data with complicated topological and geometric structures, the proposed model can successfully differentiate nearby but disjoint manifolds and intersecting manifolds with only a small amount of supervision. Moreover, this model only requires a low-complexity encoding operation, such as a locally defined linear projection. We discuss the approximation power of such networks and derive a bound that essentially depends on the intrinsic dimension of the data manifold rather than the dimension of ambient space. Next we incorporate bounds for the sampling rate of training data need to faithfully represent a given data manifold. We present numerical experiments that verify that the proposed model can effectively manage data with multi-class nearby but disjoint manifolds of different classes, overlapping manifolds, and manifolds with non-trivial topology. Finally, we conclude with some experiments on computer vision and molecular dynamics problems which showcase the efficacy of our methods on real-world data.
△ Less
Submitted 4 October, 2024; v1 submitted 22 August, 2022;
originally announced August 2022.
-
Recovery of coefficients in semilinear transport equations
Authors:
Ru-Yu Lai,
Gunther Uhlmann,
Hanming Zhou
Abstract:
We consider the inverse problem for time-dependent semilinear transport equations. We show that time-independent coefficients of both the linear (absorption or scattering coefficients) and nonlinear terms can be uniquely determined, in a stable way, from the boundary measurements by applying a linearization scheme and Carleman estimates for the linear transport equations. We establish results in b…
▽ More
We consider the inverse problem for time-dependent semilinear transport equations. We show that time-independent coefficients of both the linear (absorption or scattering coefficients) and nonlinear terms can be uniquely determined, in a stable way, from the boundary measurements by applying a linearization scheme and Carleman estimates for the linear transport equations. We establish results in both Euclidean and general geometry settings.
△ Less
Submitted 9 May, 2024; v1 submitted 20 July, 2022;
originally announced July 2022.
-
Bridging Mean-Field Games and Normalizing Flows with Trajectory Regularization
Authors:
Han Huang,
Jiajia Yu,
Jie Chen,
Rongjie Lai
Abstract:
Mean-field games (MFGs) are a modeling framework for systems with a large number of interacting agents. They have applications in economics, finance, and game theory. Normalizing flows (NFs) are a family of deep generative models that compute data likelihoods by using an invertible mapping, which is typically parameterized by using neural networks. They are useful for density modeling and data gen…
▽ More
Mean-field games (MFGs) are a modeling framework for systems with a large number of interacting agents. They have applications in economics, finance, and game theory. Normalizing flows (NFs) are a family of deep generative models that compute data likelihoods by using an invertible mapping, which is typically parameterized by using neural networks. They are useful for density modeling and data generation. While active research has been conducted on both models, few noted the relationship between the two. In this work, we unravel the connections between MFGs and NFs by contextualizing the training of an NF as solving the MFG. This is achieved by reformulating the MFG problem in terms of agent trajectories and parameterizing a discretization of the resulting MFG with flow architectures. With this connection, we explore two research directions. First, we employ expressive NF architectures to accurately solve high-dimensional MFGs, sidestepping the curse of dimensionality in traditional numerical methods. Compared with other deep learning approaches, our trajectory-based formulation encodes the continuity equation in the neural network, resulting in a better approximation of the population dynamics. Second, we regularize the training of NFs with transport costs and show the effectiveness on controlling the model's Lipschitz bound, resulting in better generalization performance. We demonstrate numerical results through comprehensive experiments on a variety of synthetic and real-life datasets.
△ Less
Submitted 29 June, 2022;
originally announced June 2022.
-
Computational Mean-field Games on Manifolds
Authors:
Jiajia Yu,
Rongjie Lai,
Wuchen Li,
Stanley Osher
Abstract:
Conventional Mean-field games/control study the behavior of a large number of rational agents moving in the Euclidean spaces. In this work, we explore the mean-field games on Riemannian manifolds. We formulate the mean-field game Nash Equilibrium on manifolds. We also establish the equivalence between the PDE system and the optimality conditions of the associated variational form on manifolds. Bas…
▽ More
Conventional Mean-field games/control study the behavior of a large number of rational agents moving in the Euclidean spaces. In this work, we explore the mean-field games on Riemannian manifolds. We formulate the mean-field game Nash Equilibrium on manifolds. We also establish the equivalence between the PDE system and the optimality conditions of the associated variational form on manifolds. Based on the triangular mesh representation of two-dimensional manifolds, we design a proximal gradient method for variational mean-field games. Our comprehensive numerical experiments on various manifolds illustrate the effectiveness and flexibility of the proposed model and numerical methods.
△ Less
Submitted 3 June, 2022;
originally announced June 2022.
-
On local antimagic vertex coloring for complete full $t$-ary trees
Authors:
Martin Bača,
Andrea Semaničová-Feňovčíková,
Ruei-Ting Lai,
Tao-Ming Wang
Abstract:
Let $G = (V, E)$ be a finite simple undirected graph without $K_2$ components. A bijection $f : E \rightarrow \{1, 2,\cdots, |E|\}$ is called a local antimagic labeling if for any two adjacent vertices $u$ and $v$, they have different vertex sums, i.e., $w(u) \neq w(v)$, where the vertex sum $w(u) = \sum_{e \in E(u)} f(e)$, and $E(u)$ is the set of edges incident to $u$. Thus any local antimagic l…
▽ More
Let $G = (V, E)$ be a finite simple undirected graph without $K_2$ components. A bijection $f : E \rightarrow \{1, 2,\cdots, |E|\}$ is called a local antimagic labeling if for any two adjacent vertices $u$ and $v$, they have different vertex sums, i.e., $w(u) \neq w(v)$, where the vertex sum $w(u) = \sum_{e \in E(u)} f(e)$, and $E(u)$ is the set of edges incident to $u$. Thus any local antimagic labeling induces a proper vertex coloring of $G$ where the vertex $v$ is assigned the color (vertex sum) $w(v)$. The local antimagic chromatic number $χ_{la}(G)$ is the minimum number of colors taken over all colorings induced by local antimagic labelings of $G$. It was conjectured \cite{Aru-Wang} that for every tree $T$ the local antimagic chromatic number $l+ 1 \leq χ_{la} ( T )\leq l+2$, where $l$ is the number of leaves of $T$. In this article we verify the above conjecture for complete full $t$-ary trees, for $t \geq 2$. A complete full $t$-ary tree is a rooted tree in which all nodes have exactly $t$ children except leaves and every leaf is of the same depth. In particular we obtain that the exact value for the local antimagic chromatic number of all complete full $t$-ary trees is $ l+1$ for odd $t$.
△ Less
Submitted 14 April, 2022; v1 submitted 9 April, 2022;
originally announced April 2022.
-
Single pixel X-ray transform and related inverse problems
Authors:
Ru-Yu Lai,
Gunther Uhlmann,
Jian Zhai,
Hanming Zhou
Abstract:
In this paper, we analyze the nonlinear single pixel X-ray transform $K$ and study the reconstruction of $f$ from the measurement $Kf$. Different from the well-known X-ray transform, the transform $K$ is a nonlinear operator and uses a single detector that integrates all rays in the space. We derive stability estimates and an inversion formula of $K$. We also consider the case where we integrate a…
▽ More
In this paper, we analyze the nonlinear single pixel X-ray transform $K$ and study the reconstruction of $f$ from the measurement $Kf$. Different from the well-known X-ray transform, the transform $K$ is a nonlinear operator and uses a single detector that integrates all rays in the space. We derive stability estimates and an inversion formula of $K$. We also consider the case where we integrate along geodesics of a Riemannian metric. Moreover, we conduct several numerical experiments to corroborate the theoretical results.
△ Less
Submitted 27 December, 2021;
originally announced December 2021.
-
Inverse transport and diffusion problems in photoacoustic imaging with nonlinear absorption
Authors:
Ru-Yu Lai,
Kui Ren,
Ting Zhou
Abstract:
Motivated by applications in imaging nonlinear optical absorption by photoacoustic tomography (PAT), we study in this work inverse coefficient problems for a semilinear radiative transport equation and its diffusion approximation with internal data that are functionals of the coefficients and the solutions to the equations. Based on the techniques of first- and second-order linearization, we deriv…
▽ More
Motivated by applications in imaging nonlinear optical absorption by photoacoustic tomography (PAT), we study in this work inverse coefficient problems for a semilinear radiative transport equation and its diffusion approximation with internal data that are functionals of the coefficients and the solutions to the equations. Based on the techniques of first- and second-order linearization, we derive uniqueness and stability results for the inverse problems. For uncertainty quantification purpose, we also establish the stability of the reconstruction of the absorption coefficients with respect to the change in the scattering coefficient.
△ Less
Submitted 16 July, 2021;
originally announced July 2021.
-
Inverse source problems in transport equations with external forces
Authors:
Ru-Yu Lai,
Hanming Zhou
Abstract:
This paper is concerned with the inverse source problem for the transport equation with external force. We show that both direct and inverse problems are uniquely solvable for generic absorption and scattering coefficients. In particular, for inverse problems, generic injectivity and a stability estimate of the source are derived. The analysis employs the Fredholm theorem and the Santalo's formula…
▽ More
This paper is concerned with the inverse source problem for the transport equation with external force. We show that both direct and inverse problems are uniquely solvable for generic absorption and scattering coefficients. In particular, for inverse problems, generic injectivity and a stability estimate of the source are derived. The analysis employs the Fredholm theorem and the Santalo's formula.
△ Less
Submitted 23 April, 2021;
originally announced April 2021.
-
An Inverse Problem for Non-linear Fractional Magnetic Schrodinger Equation
Authors:
Ru-Yu Lai,
Ting Zhou
Abstract:
In this paper, we study forward problem and inverse problem for the fractional magnetic Schrodinger equation with nonlinear electric potential. We first investigate the maximum principle for the linearized equation and apply it to show that the problem is well-posed under suitable assumptions on the exterior data. Moreover, we explore uniqueness of recovery of both magnetic and electric potentials…
▽ More
In this paper, we study forward problem and inverse problem for the fractional magnetic Schrodinger equation with nonlinear electric potential. We first investigate the maximum principle for the linearized equation and apply it to show that the problem is well-posed under suitable assumptions on the exterior data. Moreover, we explore uniqueness of recovery of both magnetic and electric potentials.
△ Less
Submitted 15 March, 2021;
originally announced March 2021.
-
A Fast Proximal Gradient Method and Convergence Analysis for Dynamic Mean Field Planning
Authors:
Jiajia Yu,
Rongjie Lai,
Wuchen Li,
Stanley Osher
Abstract:
In this paper, we propose an efficient and flexible algorithm to solve dynamic mean-field planning problems based on an accelerated proximal gradient method. Besides an easy-to-implement gradient descent step in this algorithm, a crucial projection step becomes solving an elliptic equation whose solution can be obtained by conventional methods efficiently. By induction on iterations used in the al…
▽ More
In this paper, we propose an efficient and flexible algorithm to solve dynamic mean-field planning problems based on an accelerated proximal gradient method. Besides an easy-to-implement gradient descent step in this algorithm, a crucial projection step becomes solving an elliptic equation whose solution can be obtained by conventional methods efficiently. By induction on iterations used in the algorithm, we theoretically show that the proposed discrete solution converges to the underlying continuous solution as the grid size increases. Furthermore, we generalize our algorithm to mean-field game problems and accelerate it using multilevel and multigrid strategies. We conduct comprehensive numerical experiments to confirm the convergence analysis of the proposed algorithm, to show its efficiency and mass preservation property by comparing it with state-of-the-art methods, and to illustrates its flexibility for handling various mean-field variational problems.
△ Less
Submitted 25 February, 2021;
originally announced February 2021.
-
Inverse problems for the fractional Laplace equation with lower order nonlinear perturbations
Authors:
Ru-Yu Lai,
Laurel Ohm
Abstract:
We study the inverse problem for the fractional Laplace equation with multiple nonlinear lower order terms. We show that the direct problem is well-posed and the inverse problem is uniquely solvable. More specifically, the unknown nonlinearities can be uniquely determined from exterior measurements under suitable settings.
We study the inverse problem for the fractional Laplace equation with multiple nonlinear lower order terms. We show that the direct problem is well-posed and the inverse problem is uniquely solvable. More specifically, the unknown nonlinearities can be uniquely determined from exterior measurements under suitable settings.
△ Less
Submitted 16 September, 2020;
originally announced September 2020.
-
Optimizing Mode Connectivity via Neuron Alignment
Authors:
N. Joseph Tatro,
Pin-Yu Chen,
Payel Das,
Igor Melnyk,
Prasanna Sattigeri,
Rongjie Lai
Abstract:
The loss landscapes of deep neural networks are not well understood due to their high nonconvexity. Empirically, the local minima of these loss functions can be connected by a learned curve in model space, along which the loss remains nearly constant; a feature known as mode connectivity. Yet, current curve finding algorithms do not consider the influence of symmetry in the loss surface created by…
▽ More
The loss landscapes of deep neural networks are not well understood due to their high nonconvexity. Empirically, the local minima of these loss functions can be connected by a learned curve in model space, along which the loss remains nearly constant; a feature known as mode connectivity. Yet, current curve finding algorithms do not consider the influence of symmetry in the loss surface created by model weight permutations. We propose a more general framework to investigate the effect of symmetry on landscape connectivity by accounting for the weight permutations of the networks being connected. To approximate the optimal permutation, we introduce an inexpensive heuristic referred to as neuron alignment. Neuron alignment promotes similarity between the distribution of intermediate activations of models along the curve. We provide theoretical analysis establishing the benefit of alignment to mode connectivity based on this simple heuristic. We empirically verify that the permutation given by alignment is locally optimal via a proximal alternating minimization scheme. Empirically, optimizing the weight permutation is critical for efficiently learning a simple, planar, low-loss curve between networks that successfully generalizes. Our alignment method can significantly alleviate the recently identified robust loss barrier on the path connecting two adversarial robust models and find more robust and accurate models on the path.
△ Less
Submitted 2 November, 2020; v1 submitted 4 September, 2020;
originally announced September 2020.
-
Global determination for an inverse problem from the vortex dynamics
Authors:
Ru-Yu Lai,
Hanming Zhou
Abstract:
We consider the problem of reconstructing a background potential from the dynamical behavior of vortex dipole. We prove that under suitable conditions, one can uniquely reconstruct a real-analytic potential by measuring the entrance and exit positions as well as travel times between boundary points. In particular, the work removes the flatness assumption on the potential from the earlier result. A…
▽ More
We consider the problem of reconstructing a background potential from the dynamical behavior of vortex dipole. We prove that under suitable conditions, one can uniquely reconstruct a real-analytic potential by measuring the entrance and exit positions as well as travel times between boundary points. In particular, the work removes the flatness assumption on the potential from the earlier result. A key step of our method is a constructional procedure of recovering the boundary jet of the potential.
△ Less
Submitted 10 August, 2020;
originally announced August 2020.
-
Partial Data Inverse Problems for Nonlinear Magnetic Schrödinger Equations
Authors:
Ru-Yu Lai,
Ting Zhou
Abstract:
We prove that the knowledge of the Dirichlet-to-Neumann map, measured on a part of the boundary of a bounded domain in $\mathbb{R}^n, n\geq2$, can uniquely determine, in a nonlinear magnetic Schrödinger equation, the vector-valued magnetic potential and the scalar electric potential, both being nonlinear in the solution.
We prove that the knowledge of the Dirichlet-to-Neumann map, measured on a part of the boundary of a bounded domain in $\mathbb{R}^n, n\geq2$, can uniquely determine, in a nonlinear magnetic Schrödinger equation, the vector-valued magnetic potential and the scalar electric potential, both being nonlinear in the solution.
△ Less
Submitted 5 July, 2020;
originally announced July 2020.
-
Reconstruction of the emission coefficient in the nonlinear radiative transfer equation
Authors:
Christian Klingenberg,
Ru-Yu Lai,
Qin Li
Abstract:
In this paper, we investigate an inverse problem for the radiative transfer equation that is coupled with a heat equation in a nonscattering medium in $\mathbb{R}^n$, $n\geq 2$. The two equations are coupled through a nonlinear blackbody emission term that is proportional to the fourth power of the temperature. By measuring the radiation intensity on the surface of the blackbody, we prove that the…
▽ More
In this paper, we investigate an inverse problem for the radiative transfer equation that is coupled with a heat equation in a nonscattering medium in $\mathbb{R}^n$, $n\geq 2$. The two equations are coupled through a nonlinear blackbody emission term that is proportional to the fourth power of the temperature. By measuring the radiation intensity on the surface of the blackbody, we prove that the emission property of the system can be uniquely reconstructed. In particular, we design a reconstruction procedure that uses merely one set of experiment setup to fully recover the emission parameter.
△ Less
Submitted 9 October, 2020; v1 submitted 25 June, 2020;
originally announced June 2020.
-
Inverse problems for fractional semilinear elliptic equations
Authors:
Ru-Yu Lai,
Yi-Hsuan Lin
Abstract:
This paper is concerned with the forward and inverse problems for the fractional semilinear elliptic equation $(-Δ)^s u +a(x,u)=0$ for $0<s<1$. For the forward problem, we proved the problem is well-posed and has a unique solution for small exterior data. The inverse problems we consider here consists of two cases. First we demonstrate that an unknown coefficient $a(x,u)$ can be uniquely determine…
▽ More
This paper is concerned with the forward and inverse problems for the fractional semilinear elliptic equation $(-Δ)^s u +a(x,u)=0$ for $0<s<1$. For the forward problem, we proved the problem is well-posed and has a unique solution for small exterior data. The inverse problems we consider here consists of two cases. First we demonstrate that an unknown coefficient $a(x,u)$ can be uniquely determined from the knowledge of exterior measurements, known as the Dirichlet-to-Neumann map. Second, despite the presence of an unknown obstacle in the media, we show that the obstacle and the coefficient can be recovered concurrently from these measurements. Finally, we investigate that these two fractional inverse problems can also be solved by using a single measurement, and all results hold for any dimension $n\geq 1$.
△ Less
Submitted 1 April, 2020;
originally announced April 2020.
-
Reconstruction of the collision kernel in the nonlinear Boltzmann equation
Authors:
Ru-Yu Lai,
Gunther Uhlmann,
Yang Yang
Abstract:
We consider an inverse problem for the Boltzmann equation with nonlinear collision operator in dimensions $n\geq 2$. We show that the kinetic collision kernel can be uniquely determined from the incoming-to-outgoing mappings on the boundary of the domain provided that the kernel satisfies a monotonicity condition. Furthermore, a reconstruction formula is also derived. The key methodology is based…
▽ More
We consider an inverse problem for the Boltzmann equation with nonlinear collision operator in dimensions $n\geq 2$. We show that the kinetic collision kernel can be uniquely determined from the incoming-to-outgoing mappings on the boundary of the domain provided that the kernel satisfies a monotonicity condition. Furthermore, a reconstruction formula is also derived. The key methodology is based on the higher-order linearization scheme to reduce a nonlinear equation into simpler linear equations by introducing multiple small parameters into the original equation.
△ Less
Submitted 20 March, 2020;
originally announced March 2020.
-
Generalized Unnormalized Optimal Transport and its fast algorithms
Authors:
Wonjun Lee,
Rongjie Lai,
Wuchen Li,
Stanley Osher
Abstract:
We introduce fast algorithms for generalized unnormalized optimal transport. To handle densities with different total mass, we consider a dynamic model, which mixes the $L^p$ optimal transport with $L^p$ distance. For $p=1$, we derive the corresponding $L^1$ generalized unnormalized Kantorovich formula. We further show that the problem becomes a simple $L^1$ minimization which is solved efficientl…
▽ More
We introduce fast algorithms for generalized unnormalized optimal transport. To handle densities with different total mass, we consider a dynamic model, which mixes the $L^p$ optimal transport with $L^p$ distance. For $p=1$, we derive the corresponding $L^1$ generalized unnormalized Kantorovich formula. We further show that the problem becomes a simple $L^1$ minimization which is solved efficiently by a primal-dual algorithm. For $p=2$, we derive the $L^2$ generalized unnormalized Kantorovich formula, a new unnormalized Monge problem and the corresponding Monge-Ampère equation. Furthermore, we introduce a new unconstrained optimization formulation of the problem. The associated gradient flow is essentially related to an elliptic equation which can be solved efficiently. Here the proposed gradient descent procedure together with the Nesterov acceleration involves the Hamilton-Jacobi equation which arises from the KKT conditions. Several numerical examples are presented to illustrate the effectiveness of the proposed algorithms.
△ Less
Submitted 30 January, 2020;
originally announced January 2020.
-
On diffusive scaling in acousto-optic imaging
Authors:
Francis J. Chung,
Ru-Yu Lai,
Qin Li
Abstract:
Acousto-optic imaging (AOI) is a hybrid imaging process. By perturbing the to-be-reconstructed tissues with acoustic waves, one introduces the interaction between the acoustic and optical waves, leading to a more stable reconstruction of the optical properties. The mathematical model was described in [25], with the radiative transfer equation serving as the forward model for the optical transport.…
▽ More
Acousto-optic imaging (AOI) is a hybrid imaging process. By perturbing the to-be-reconstructed tissues with acoustic waves, one introduces the interaction between the acoustic and optical waves, leading to a more stable reconstruction of the optical properties. The mathematical model was described in [25], with the radiative transfer equation serving as the forward model for the optical transport. In this paper we investigate the stability of the reconstruction. In particular, we are interested in how the stability depends on the Knudsen number, Kn, a quantity that measures the intensity of the scattering effect of photon particles in a media. Our analysis shows that as Kn decreases to zero, photons scatter more frequently, and since information is lost, the reconstruction becomes harder. To counter this effect, devices need to be constructed so that laser beam is highly concentrated. We will give a quantitative error bound, and explicitly show that such concentration has an exponential dependence on Kn. Numerical evidence will be provided to verify the proof.
△ Less
Submitted 5 May, 2020; v1 submitted 21 January, 2020;
originally announced January 2020.
-
The Calderón problem for a space-time fractional parabolic equation
Authors:
Ru-Yu Lai,
Yi-Hsuan Lin,
Angkana Rüland
Abstract:
In this article we study an inverse problem for the space-time fractional parabolic operator $(\partial_t-Δ)^s+Q$ with $0<s<1$ in any space dimension. We uniquely determine the unknown bounded potential $Q$ from infinitely many exterior Dirichlet-to-Neumann type measurements. This relies on Runge approximation and the dual global weak unique continuation properties of the equation under considerat…
▽ More
In this article we study an inverse problem for the space-time fractional parabolic operator $(\partial_t-Δ)^s+Q$ with $0<s<1$ in any space dimension. We uniquely determine the unknown bounded potential $Q$ from infinitely many exterior Dirichlet-to-Neumann type measurements. This relies on Runge approximation and the dual global weak unique continuation properties of the equation under consideration. In discussing weak unique continuation of our operator, a main feature of our argument relies on a Carleman estimate for the associated fractional parabolic Caffarelli-Silvestre extension. Furthermore, we also discuss constructive single measurement results based on the approximation and unique continuation properties of the equation.
△ Less
Submitted 21 May, 2019;
originally announced May 2019.
-
Parameter Reconstruction for general transport equation
Authors:
Ru-Yu Lai,
Qin Li
Abstract:
We consider the inverse problem for the general transport equation with external field, source term and absorption coefficient. We show that the source and the absorption coefficients can be uniquely reconstructed from the boundary measurement, in a Lipschitz stable manner. Specifically, the uniqueness and stability are obtained by using the Carleman estimate in which a special weight function is…
▽ More
We consider the inverse problem for the general transport equation with external field, source term and absorption coefficient. We show that the source and the absorption coefficients can be uniquely reconstructed from the boundary measurement, in a Lipschitz stable manner. Specifically, the uniqueness and stability are obtained by using the Carleman estimate in which a special weight function is designed to pick up information on the desired parameter.
△ Less
Submitted 22 April, 2019;
originally announced April 2019.
-
Boundary determination of electromagnetic and Lamé parameters with corrupted data
Authors:
Pedro Caro,
Ru-Yu Lai,
Yi-Hsuan Lin,
Ting Zhou
Abstract:
We study boundary determination for an inverse problem associated to the time-harmonic Maxwell equations and another associated to the isotropic elasticity system. We identify the electromagnetic parameters and the Lamé moduli for these two systems from the corresponding boundary measurements. In a first step we reconstruct Lipschitz magnetic permeability, electric permittivity and conductivity on…
▽ More
We study boundary determination for an inverse problem associated to the time-harmonic Maxwell equations and another associated to the isotropic elasticity system. We identify the electromagnetic parameters and the Lamé moduli for these two systems from the corresponding boundary measurements. In a first step we reconstruct Lipschitz magnetic permeability, electric permittivity and conductivity on the surface from the ideal boundary measurements. Then, we study inverse problems for Maxwell equations and the isotropic elasticity system assuming that the data contains measurement errors. For both systems, we provide explicit formulas to reconstruct the parameters on the boundary as well as its rate of convergence formula.
△ Less
Submitted 8 March, 2019;
originally announced March 2019.
-
Low-rank Matrix Completion in a General Non-orthogonal Basis
Authors:
Abiy Tasissa,
Rongjie Lai
Abstract:
This paper considers theoretical analysis of recovering a low rank matrix given a few expansion coefficients with respect to any basis. The current approach generalizes the existing analysis for the low-rank matrix completion problem with sampling under entry sensing or with respect to a symmetric orthonormal basis. The analysis is based on dual certificates using a dual basis approach and does no…
▽ More
This paper considers theoretical analysis of recovering a low rank matrix given a few expansion coefficients with respect to any basis. The current approach generalizes the existing analysis for the low-rank matrix completion problem with sampling under entry sensing or with respect to a symmetric orthonormal basis. The analysis is based on dual certificates using a dual basis approach and does not assume the restricted isometry property (RIP). We introduce a condition on the basis called the correlation condition. This condition can be computed in time $O(n^3)$ and holds for many cases of deterministic basis where RIP might not hold or is NP hard to verify. If the correlation condition holds and the underlying low rank matrix obeys the coherence condition with parameter $ν$, under additional mild assumptions, our main result shows that the true matrix can be recovered with very high probability from $O(nrν\log^2n)$ uniformly random expansion coefficients.
△ Less
Submitted 14 December, 2018;
originally announced December 2018.
-
Nonisometric Surface Registration via Conformal Laplace-Beltrami Basis Pursuit
Authors:
Stefan C. Schonsheck,
Michael M. Bronstein,
Rongjie Lai
Abstract:
Surface registration is one of the most fundamental problems in geometry processing. Many approaches have been developed to tackle this problem in cases where the surfaces are nearly isometric. However, it is much more challenging to compute correspondence between surfaces which are intrinsically less similar. In this paper, we propose a variational model to align the Laplace-Beltrami (LB) eigensy…
▽ More
Surface registration is one of the most fundamental problems in geometry processing. Many approaches have been developed to tackle this problem in cases where the surfaces are nearly isometric. However, it is much more challenging to compute correspondence between surfaces which are intrinsically less similar. In this paper, we propose a variational model to align the Laplace-Beltrami (LB) eigensytems of two non-isometric genus zero shapes via conformal deformations. This method enables us compute to geometric meaningful point-to-point maps between non-isometric shapes. Our model is based on a novel basis pursuit scheme whereby we simultaneously compute a conformal deformation of a 'target shape' and its deformed LB eigensytem. We solve the model using an proximal alternating minimization algorithm hybridized with the augmented Lagrangian method which produces accurate correspondences given only a few landmark points. We also propose a reinitialization scheme to overcome some of the difficulties caused by the non-convexity of the variational problem. Intensive numerical experiments illustrate the effectiveness and robustness of the proposed method to handle non-isometric surfaces with large deformation with respect to both noise on the underlying manifolds and errors within the given landmarks.
△ Less
Submitted 19 September, 2018;
originally announced September 2018.
-
Inverse problems for the stationary transport equation in the diffusion scaling
Authors:
Ru-Yu Lai,
Qin Li,
Gunther Uhlmann
Abstract:
We consider the inverse problem of reconstructing the optical parameters of the radiative transfer equation (RTE) from boundary measurements in the diffusion limit. In the diffusive regime (the Knudsen number $\mathsf{Kn}\ll 1$), the forward problem for the stationary RTE is well approximated by an elliptic equation. However, the connection between the inverse problem for the RTE and the inverse p…
▽ More
We consider the inverse problem of reconstructing the optical parameters of the radiative transfer equation (RTE) from boundary measurements in the diffusion limit. In the diffusive regime (the Knudsen number $\mathsf{Kn}\ll 1$), the forward problem for the stationary RTE is well approximated by an elliptic equation. However, the connection between the inverse problem for the RTE and the inverse problem for the elliptic equation has not been fully developed. This problem is particularly interesting because the former one is mildly ill-posed , with a Lipschitz type stability estimate, while the latter is well known to be severely ill-posed with a logarithmic type stability estimate. In this paper, we derive stability estimates for the inverse problem for RTE and examine its dependence on $\mathsf{Kn}$. We show that the stability is Lipschitz in all regimes, but the coefficient deteriorates as $e^{\frac{1}{\mathsf{Kn}}}$, making the inverse problem of RTE severely ill-posed when $\mathsf{Kn}$ is small. In this way we connect the two inverse problems. Numerical results agree with the analysis of worsening stability as the Knudsen number gets smaller.
△ Less
Submitted 6 August, 2018;
originally announced August 2018.
-
Parallel Transport Convolution: A New Tool for Convolutional Neural Networks on Manifolds
Authors:
Stefan C. Schonsheck,
Bin Dong,
Rongjie Lai
Abstract:
Convolution has been playing a prominent role in various applications in science and engineering for many years. It is the most important operation in convolutional neural networks. There has been a recent growth of interests of research in generalizing convolutions on curved domains such as manifolds and graphs. However, existing approaches cannot preserve all the desirable properties of Euclidea…
▽ More
Convolution has been playing a prominent role in various applications in science and engineering for many years. It is the most important operation in convolutional neural networks. There has been a recent growth of interests of research in generalizing convolutions on curved domains such as manifolds and graphs. However, existing approaches cannot preserve all the desirable properties of Euclidean convolutions, namely compactly supported filters, directionality, transferability across different manifolds. In this paper we develop a new generalization of the convolution operation, referred to as parallel transport convolution (PTC), on Riemannian manifolds and their discrete counterparts. PTC is designed based on the parallel transportation which is able to translate information along a manifold and to intrinsically preserve directionality. PTC allows for the construction of compactly supported filters and is also robust to manifold deformations. This enables us to preform wavelet-like operations and to define deep convolutional neural networks on curved domains.
△ Less
Submitted 8 December, 2018; v1 submitted 20 May, 2018;
originally announced May 2018.
-
Exact Reconstruction of Euclidean Distance Geometry Problem Using Low-rank Matrix Completion
Authors:
Abiy Tasissa,
Rongjie Lai
Abstract:
The Euclidean distance geometry problem arises in a wide variety of applications, from determining molecular conformations in computational chemistry to localization in sensor networks. When the distance information is incomplete, the problem can be formulated as a nuclear norm minimization problem. In this paper, this minimization program is recast as a matrix completion problem of a low-rank…
▽ More
The Euclidean distance geometry problem arises in a wide variety of applications, from determining molecular conformations in computational chemistry to localization in sensor networks. When the distance information is incomplete, the problem can be formulated as a nuclear norm minimization problem. In this paper, this minimization program is recast as a matrix completion problem of a low-rank $r$ Gram matrix with respect to a suitable basis. The well known restricted isometry property can not be satisfied in this scenario. Instead, a dual basis approach is introduced to theoretically analyze the reconstruction problem. If the Gram matrix satisfies certain coherence conditions with parameter $ν$, the main result shows that the underlying configuration of $n$ points can be recovered with very high probability from $O(nrν\log^{2}(n))$ uniformly random samples. Computationally, simple and fast algorithms are designed to solve the Euclidean distance geometry problem. Numerical tests on different three dimensional data and protein molecules validate effectiveness and efficiency of the proposed algorithms.
△ Less
Submitted 28 October, 2018; v1 submitted 12 April, 2018;
originally announced April 2018.
-
Global uniqueness for the semilinear fractional Schrödinger equation
Authors:
Ru-Yu Lai,
Yi-Hsuan Lin
Abstract:
We study global uniqueness in an inverse problem for the fractional semilinear Schrödinger equation $(-Δ)^{s}u+q(x,u)=0$ with $s\in (0,1)$. We show that an unknown function $q(x,u)$ can be uniquely determined by the Cauchy data set. In particular, this result holds for any space dimension greater than or equal to $2$. Moreover, we demonstrate the comparison principle and provide a $L^\infty$ estim…
▽ More
We study global uniqueness in an inverse problem for the fractional semilinear Schrödinger equation $(-Δ)^{s}u+q(x,u)=0$ with $s\in (0,1)$. We show that an unknown function $q(x,u)$ can be uniquely determined by the Cauchy data set. In particular, this result holds for any space dimension greater than or equal to $2$. Moreover, we demonstrate the comparison principle and provide a $L^\infty$ estimate for this nonlocal equation under appropriate regularity assumptions.
△ Less
Submitted 19 October, 2017;
originally announced October 2017.
-
Quench detection on a superconducting radio-frequency cavity
Authors:
Ru-Yu Lai,
Daniel Spirn
Abstract:
We study quench detection in superconducting accelerator cavities cooled with He-II. A rigorous mathematical formula is derived to localize the quench position from dynamical data over a finite time interval at a second sound detector.
We study quench detection in superconducting accelerator cavities cooled with He-II. A rigorous mathematical formula is derived to localize the quench position from dynamical data over a finite time interval at a second sound detector.
△ Less
Submitted 17 November, 2017; v1 submitted 12 October, 2017;
originally announced October 2017.
-
Global Optimization with Orthogonality Constraints via Stochastic Diffusion on Manifold
Authors:
Honglin Yuan,
Xiaoyi Gu,
Rongjie Lai,
Zaiwen Wen
Abstract:
Orthogonality constrained optimization is widely used in applications from science and engineering. Due to the nonconvex orthogonality constraints, many numerical algorithms often can hardly achieve the global optimality. We aim at establishing an efficient scheme for finding global minimizers under one or more orthogonality constraints. The main concept is based on noisy gradient flow constructed…
▽ More
Orthogonality constrained optimization is widely used in applications from science and engineering. Due to the nonconvex orthogonality constraints, many numerical algorithms often can hardly achieve the global optimality. We aim at establishing an efficient scheme for finding global minimizers under one or more orthogonality constraints. The main concept is based on noisy gradient flow constructed from stochastic differential equations (SDE) on the Stiefel manifold, the differential geometric characterization of orthogonality constraints. We derive an explicit representation of SDE on the Stiefel manifold endowed with a canonical metric and propose a numerically efficient scheme to simulate this SDE based on Cayley transformation with theoretical convergence guarantee. The convergence to global optimizers is proved under second-order continuity. The effectiveness and efficiency of the proposed algorithms are demonstrated on a variety of problems including homogeneous polynomial optimization, computation of stability number, and 3D structure determination from Common Lines in Cryo-EM.
△ Less
Submitted 7 July, 2017;
originally announced July 2017.
-
Optimal Impulse Control of a Simple Reparable System in a Nonreflexive Banach Space
Authors:
Weiwei Hu,
Rongjie Lai,
Houbao Xu,
Chuang Zheng
Abstract:
We discuss the problem of optimal impulse control representing the preventive maintenance of a simple reparable system. The system model is governed by coupled transport and integro-differential equations in a nonreflexive Banach space. The objective of this paper is to construct nonnegative impulse control inputs at given system running times that minimize the probability of the system in failure…
▽ More
We discuss the problem of optimal impulse control representing the preventive maintenance of a simple reparable system. The system model is governed by coupled transport and integro-differential equations in a nonreflexive Banach space. The objective of this paper is to construct nonnegative impulse control inputs at given system running times that minimize the probability of the system in failure mode. To guarantee the nonnegativity of the controlled system, we consider the control inputs to depend on the system state. This essentially leads to a bilinear control problem. We first present a rigorous proof of existence of an optimal controller and then apply the variational inequality to derive the first-order necessary conditions of optimality.
△ Less
Submitted 27 March, 2017;
originally announced March 2017.
-
Point cloud discretization of Fokker-Planck operators for committor functions
Authors:
Rongjie Lai,
Jianfeng Lu
Abstract:
The committor functions provide useful information to the understanding of transitions of a stochastic system between disjoint regions in phase space. In this work, we develop a point cloud discretization for Fokker-Planck operators to numerically calculate the committor function, with the assumption that the transition occurs on an intrinsically low-dimensional manifold in the ambient potentially…
▽ More
The committor functions provide useful information to the understanding of transitions of a stochastic system between disjoint regions in phase space. In this work, we develop a point cloud discretization for Fokker-Planck operators to numerically calculate the committor function, with the assumption that the transition occurs on an intrinsically low-dimensional manifold in the ambient potentially high dimensional configurational space of the stochastic system. Numerical examples on model systems validate the effectiveness of the proposed method.
△ Less
Submitted 27 March, 2017;
originally announced March 2017.
-
Manifold Based Low-rank Regularization for Image Restoration and Semi-supervised Learning
Authors:
Rongjie Lai,
Jia Li
Abstract:
Low-rank structures play important role in recent advances of many problems in image science and data science. As a natural extension of low-rank structures for data with nonlinear structures, the concept of the low-dimensional manifold structure has been considered in many data processing problems. Inspired by this concept, we consider a manifold based low-rank regularization as a linear approxim…
▽ More
Low-rank structures play important role in recent advances of many problems in image science and data science. As a natural extension of low-rank structures for data with nonlinear structures, the concept of the low-dimensional manifold structure has been considered in many data processing problems. Inspired by this concept, we consider a manifold based low-rank regularization as a linear approximation of manifold dimension. This regularization is less restricted than the global low-rank regularization, and thus enjoy more flexibility to handle data with nonlinear structures. As applications, we demonstrate the proposed regularization to classical inverse problems in image sciences and data sciences including image inpainting, image super-resolution, X-ray computer tomography (CT) image reconstruction and semi-supervised learning. We conduct intensive numerical experiments in several image restoration problems and a semi-supervised learning problem of classifying handwritten digits using the MINST data. Our numerical tests demonstrate the effectiveness of the proposed methods and illustrate that the new regularization methods produce outstanding results by comparing with many existing methods.
△ Less
Submitted 8 February, 2017;
originally announced February 2017.
-
Solving Partial Differential Equations on Manifolds From Incomplete Inter-Point Distance
Authors:
Rongjie Lai,
Jia Li
Abstract:
Solutions of partial differential equations (PDEs) on manifolds have provided important applications in different fields in science and engineering. Existing methods are majorly based on discretization of manifolds as implicit functions, triangle meshes, or point clouds, where the manifold structure is approximated by either zero level set of an implicit function or a set of points. In many applic…
▽ More
Solutions of partial differential equations (PDEs) on manifolds have provided important applications in different fields in science and engineering. Existing methods are majorly based on discretization of manifolds as implicit functions, triangle meshes, or point clouds, where the manifold structure is approximated by either zero level set of an implicit function or a set of points. In many applications, manifolds might be only provided as an inter-point distance matrix with possible missing values. This paper discusses a framework to discretize PDEs on manifolds represented as incomplete inter-point distance information. Without conducting a time-consuming global coordinates reconstruction, we propose a more efficient strategy by discretizing differential operators only based on point-wisely local reconstruction. Our local reconstruction model is based on the recent advances of low-rank matrix completion theory, where only a very small random portion of distance information is required. This method enables us to conduct analyses of incomplete distance data using solutions of special designed PDEs such as the Laplace-Beltrami (LB) eigen-system. As an application, we demonstrate a new way of manifold reconstruction from an incomplete distance by stitching patches using the spectrum of the LB operator. Intensive numerical experiments demonstrate the effectiveness of the proposed methods.
△ Less
Submitted 1 August, 2017; v1 submitted 10 January, 2017;
originally announced January 2017.
-
Nonparaxial Near-nondiffracting Accelerating Optical Beams
Authors:
Ru-Yu Lai,
Ting Zhou
Abstract:
We show that new families of accelerating and almost nondiffracting beams (solutions) for Maxwell's equations can be constructed. These are complex geometrical optics (CGO) solutions to Maxwell's equations with nonlinear limiting Carleman weights. They have the form of wave packets that propagate along circular trajectories while almost preserving a transverse intensity profile. We also show simil…
▽ More
We show that new families of accelerating and almost nondiffracting beams (solutions) for Maxwell's equations can be constructed. These are complex geometrical optics (CGO) solutions to Maxwell's equations with nonlinear limiting Carleman weights. They have the form of wave packets that propagate along circular trajectories while almost preserving a transverse intensity profile. We also show similar waves constructed using the approach combining CGO solutions and the Kelvin transform.
△ Less
Submitted 5 August, 2016;
originally announced August 2016.
-
An inverse problem from condense matter physics
Authors:
Ru-Yu Lai,
Ravi Shankar,
Daniel Spirn,
Gunther Uhlmann
Abstract:
We consider the problem of reconstructing the features of a weak anisotropic background potential by the trajectories of vortex dipoles in a nonlinear Gross-Pitaevskii equation. At leading order, the dynamics of vortex dipoles are given by a Hamiltonian system. If the background potential is sufficiently smooth and flat, the background can be reconstructed using ideas from the boundary and the len…
▽ More
We consider the problem of reconstructing the features of a weak anisotropic background potential by the trajectories of vortex dipoles in a nonlinear Gross-Pitaevskii equation. At leading order, the dynamics of vortex dipoles are given by a Hamiltonian system. If the background potential is sufficiently smooth and flat, the background can be reconstructed using ideas from the boundary and the lens rigidity problems. We prove that reconstructions are unique, derive an approximate reconstruction formula, and present numerical examples.
△ Less
Submitted 25 September, 2017; v1 submitted 23 June, 2016;
originally announced June 2016.
-
Applications of CGO Solutions on Coupled-Physics Inverse Problems
Authors:
Ilker Kocyigit,
Ru-Yu Lai,
Lingyun Qiu,
Yang Yang,
Ting Zhou
Abstract:
This paper surveys inverse problems arising in several coupled-physics imaging modalities for both medical and geophysical purposes. These include Photo-acoustic Tomography (PAT), Thermo-acoustic Tomography (TAT), Electro-Seismic Conversion, Transient Elastrography (TE) and Acousto-Electric Tomography (AET). These inverse problems typically consists of multiple inverse steps, each of which corresp…
▽ More
This paper surveys inverse problems arising in several coupled-physics imaging modalities for both medical and geophysical purposes. These include Photo-acoustic Tomography (PAT), Thermo-acoustic Tomography (TAT), Electro-Seismic Conversion, Transient Elastrography (TE) and Acousto-Electric Tomography (AET). These inverse problems typically consists of multiple inverse steps, each of which corresponds to one of the wave propagations involved. The review focus on those steps known as the inverse problems with internal data, in which the complex geometrical optics (CGO) solutions to the underlying equations turn out to be useful in showing the uniqueness and stability in determining the desired information.
△ Less
Submitted 11 August, 2016; v1 submitted 21 December, 2015;
originally announced December 2015.
-
Localized density matrix minimization and linear scaling algorithms
Authors:
Rongjie Lai,
Jianfeng Lu
Abstract:
We propose a convex variational approach to compute localized density matrices for both zero temperature and finite temperature cases, by adding an entry-wise $\ell_1$ regularization to the free energy of the quantum system. Based on the fact that the density matrix decays exponential away from the diagonal for insulating system or system at finite temperature, the proposed $\ell_1$ regularized va…
▽ More
We propose a convex variational approach to compute localized density matrices for both zero temperature and finite temperature cases, by adding an entry-wise $\ell_1$ regularization to the free energy of the quantum system. Based on the fact that the density matrix decays exponential away from the diagonal for insulating system or system at finite temperature, the proposed $\ell_1$ regularized variational method provides a nice way to approximate the original quantum system. We provide theoretical analysis of the approximation behavior and also design convergence guaranteed numerical algorithms based on Bregman iteration. More importantly, the $\ell_1$ regularized system naturally leads to localized density matrices with banded structure, which enables us to develop approximating algorithms to find the localized density matrices with computation cost linearly dependent on the problem size.
△ Less
Submitted 2 June, 2015;
originally announced June 2015.
-
Increasing stability for the conductivity and attenuation coefficients
Authors:
Ru-Yu Lai,
Victor Isakov,
Jenn-Nan Wang
Abstract:
In this work we consider stability of recovery of the conductivity and attenuation coefficients of the stationary Maxwell and Schrödinger equations from a complete set of (Cauchy) boundary data. By using complex geometrical optics solutions we derive some bounds which can be viewed as an evidence of increasing stability in these inverse problems when frequency is growing.
In this work we consider stability of recovery of the conductivity and attenuation coefficients of the stationary Maxwell and Schrödinger equations from a complete set of (Cauchy) boundary data. By using complex geometrical optics solutions we derive some bounds which can be viewed as an evidence of increasing stability in these inverse problems when frequency is growing.
△ Less
Submitted 1 May, 2015;
originally announced May 2015.
-
Multi-scale Non-Rigid Point Cloud Registration Using Robust Sliced-Wasserstein Distance via Laplace-Beltrami Eigenmap
Authors:
Rongjie Lai,
Hongkai Zhao
Abstract:
In this work, we propose computational models and algorithms for point cloud registration with non-rigid transformation. First, point clouds sampled from manifolds originally embedded in some Euclidean space $\mathbb{R}^D$ are transformed to new point clouds embedded in $\mathbb{R}^n$ by Laplace-Beltrami(LB) eigenmap using the $n$ leading eigenvalues and corresponding eigenfunctions of LB operator…
▽ More
In this work, we propose computational models and algorithms for point cloud registration with non-rigid transformation. First, point clouds sampled from manifolds originally embedded in some Euclidean space $\mathbb{R}^D$ are transformed to new point clouds embedded in $\mathbb{R}^n$ by Laplace-Beltrami(LB) eigenmap using the $n$ leading eigenvalues and corresponding eigenfunctions of LB operator defined intrinsically on the manifolds. The LB eigenmap are invariant under isometric transformation of the original manifolds. Then we design computational models and algorithms for registration of the transformed point clouds in distribution/probability form based on the optimal transport theory which provides both generality and flexibility to handle general point clouds setting. Our methods use robust sliced-Wasserstein distance, which is as the average of projected Wasserstein distance along different directions, and incorporate a rigid transformation to handle ambiguities introduced by the Laplace-Beltrami eigenmap. By going from smaller $n$, which provides a quick and robust registration (based on coarse scale features) as well as a good initial guess for finer scale registration, to a larger $n$, our method provides an efficient, robust and accurate approach for multi-scale non-rigid point cloud registration.
△ Less
Submitted 14 June, 2014;
originally announced June 2014.
-
Maximization of Laplace-Beltrami eigenvalues on closed Riemannian surfaces
Authors:
Chiu-Yen Kao,
Rongjie Lai,
Braxton Osting
Abstract:
Let $(M,g)$ be a connected, closed, orientable Riemannian surface and denote by $λ_k(M,g)$ the $k$-th eigenvalue of the Laplace-Beltrami operator on $(M,g)$. In this paper, we consider the mapping $(M, g)\mapsto λ_k(M,g)$. We propose a computational method for finding the conformal spectrum $Λ^c_k(M,[g_0])$, which is defined by the eigenvalue optimization problem of maximizing $λ_k(M,g)$ for $k$ f…
▽ More
Let $(M,g)$ be a connected, closed, orientable Riemannian surface and denote by $λ_k(M,g)$ the $k$-th eigenvalue of the Laplace-Beltrami operator on $(M,g)$. In this paper, we consider the mapping $(M, g)\mapsto λ_k(M,g)$. We propose a computational method for finding the conformal spectrum $Λ^c_k(M,[g_0])$, which is defined by the eigenvalue optimization problem of maximizing $λ_k(M,g)$ for $k$ fixed as $g$ varies within a conformal class $[g_0]$ of fixed volume $textrm{vol}(M,g) = 1$. We also propose a computational method for the problem where $M$ is additionally allowed to vary over surfaces with fixed genus, $γ$. This is known as the topological spectrum for genus $γ$ and denoted by $Λ^t_k(γ)$. Our computations support a conjecture of N. Nadirashvili (2002) that $Λ^t_k(0) = 8 πk$, attained by a sequence of surfaces degenerating to a union of $k$ identical round spheres. Furthermore, based on our computations, we conjecture that $Λ^t_k(1) = \frac{8π^2}{\sqrt{3}} + 8π(k-1)$, attained by a sequence of surfaces degenerating into a union of an equilateral flat torus and $k-1$ identical round spheres. The values are compared to several surfaces where the Laplace-Beltrami eigenvalues are well-known, including spheres, flat tori, and embedded tori. In particular, we show that among flat tori of volume one, the $k$-th Laplace-Beltrami eigenvalue has a local maximum with value $λ_k = 4π^2 \left\lceil \frac{k}{2} \right\rceil^2 \left( \left\lceil \frac{k}{2} \right\rceil^2 - \frac{1}{4}\right)^{-\frac{1}{2}}$. Several properties are also studied computationally, including uniqueness, symmetry, and eigenvalue multiplicity.
△ Less
Submitted 25 March, 2016; v1 submitted 19 May, 2014;
originally announced May 2014.
