-
Enhancing Accuracy in Differentially Private Distributed Optimization Through Sensitivity Reduction
Authors:
Furan Xie,
Bing Liu,
Li Chai
Abstract:
In this paper, we investigate the problem of differentially private distributed optimization. Recognizing that lower sensitivity leads to higher accuracy, we analyze the key factors influencing the sensitivity of differentially private distributed algorithms. Building on these insights, we propose a novel differentially private distributed algorithm that enhances optimization accuracy by reducing…
▽ More
In this paper, we investigate the problem of differentially private distributed optimization. Recognizing that lower sensitivity leads to higher accuracy, we analyze the key factors influencing the sensitivity of differentially private distributed algorithms. Building on these insights, we propose a novel differentially private distributed algorithm that enhances optimization accuracy by reducing sensitivity. To ensure practical applicability, we derive a closed-form expression for the noise parameter as a function of the privacy budget. Furthermore, we rigorously prove that the proposed algorithm can achieve arbitrarily rigorous $ε$-differential privacy, establish its convergence in the mean square sense, and provide an upper bound on its optimization accuracy. Finally, extensive comparisons with various privacy-preserving methods validate the effectiveness of our algorithm.
△ Less
Submitted 12 May, 2025;
originally announced May 2025.
-
Frozen Gaussian Grid-point Correction For Semi-classical Schrödinger Equation
Authors:
Lihui Chai,
Zili Deng
Abstract:
We propose an efficient reconstruction algorithm named the frozen Gaussian grid-point correction (FGGC) for computing the Schrödinger equation in the semi-classical regime using the frozen Gaussian approximation (FGA). The FGA has demonstrated its superior efficiency in dealing with semi-classical problems and high-frequency wave propagations. However, reconstructing the wave function from a large…
▽ More
We propose an efficient reconstruction algorithm named the frozen Gaussian grid-point correction (FGGC) for computing the Schrödinger equation in the semi-classical regime using the frozen Gaussian approximation (FGA). The FGA has demonstrated its superior efficiency in dealing with semi-classical problems and high-frequency wave propagations. However, reconstructing the wave function from a large number of Gaussian wave-packets is typically computationally intensive. This difficulty arises because these wave-packets propagate along the FGA trajectories to non-grid positions, making the application of the fast Fourier transform infeasible. In this work, we introduce the concept of ``on-grid correction'' and derive the formulas for the least squares approximation of Gaussian wave-packets, and also provide a detailed process of the FGGC algorithm. Furthermore, we rigorously prove that the error introduced by the least squares approximation on each Gaussian wave-packet is independent of the semi-classical parameter $\varepsilon$. Numerical experiments show that the FGGC algorithm can significantly improve reconstruction efficiency while introducing only negligible error, making it a powerful tool for solving the semi-classical Schrödinger equation, especially in applications requiring both accuracy and efficiency.
△ Less
Submitted 30 April, 2025;
originally announced April 2025.
-
Distributed Bilevel Optimization via Adaptive Penalization with Time-Scale Separation
Authors:
Youcheng Niu,
Jinming Xu,
Ying Sun,
Li Chai,
Jiming Chen
Abstract:
This paper studies a class of distributed bilevel optimization (DBO) problems with a coupled inner-level subproblem. Existing approaches typically rely on hypergradient estimations involving computationally expensive Hessian information. To address this, we propose an equivalent constrained reformulation by treating the inner-level subproblem as an inequality constraint, and introduce an adaptive…
▽ More
This paper studies a class of distributed bilevel optimization (DBO) problems with a coupled inner-level subproblem. Existing approaches typically rely on hypergradient estimations involving computationally expensive Hessian information. To address this, we propose an equivalent constrained reformulation by treating the inner-level subproblem as an inequality constraint, and introduce an adaptive penalty function to properly penalize both inequality and consensus constraints based on subproblem properties. Moreover, we propose a loopless distributed algorithm, \ALGNAME, that employs multiple-timescale updates to solve each subproblem asymptotically without requiring Hessian information. Theoretically, we establish convergence rates of $\mathcal{O}(\frac{κ^4}{(1-ρ)^2 K^{1/3}})$ for nonconvex-strongly-convex cases and $\mathcal{O}(\frac{κ^2}{(1-ρ)^2 K^{2/3}})$ for distributed min-max problems. Our analysis shows the clear dependence of convergence performance on bilevel heterogeneity, the adaptive penalty parameter, and network connectivity, with a weaker assumption on heterogeneity requiring only bounded first-order heterogeneity at the optimum. Numerical experiments validate our theoretical findings.
△ Less
Submitted 15 December, 2024;
originally announced December 2024.
-
Error estimates of physics-informed neural networks for approximating Boltzmann equation
Authors:
Elie Abdo,
Lihui Chai,
Ruimeng Hu,
Xu Yang
Abstract:
Motivated by the recent successful application of physics-informed neural networks (PINNs) to solve Boltzmann-type equations [S. Jin, Z. Ma, and K. Wu, J. Sci. Comput., 94 (2023), pp. 57], we provide a rigorous error analysis for PINNs in approximating the solution of the Boltzmann equation near a global Maxwellian. The challenge arises from the nonlocal quadratic interaction term defined in the u…
▽ More
Motivated by the recent successful application of physics-informed neural networks (PINNs) to solve Boltzmann-type equations [S. Jin, Z. Ma, and K. Wu, J. Sci. Comput., 94 (2023), pp. 57], we provide a rigorous error analysis for PINNs in approximating the solution of the Boltzmann equation near a global Maxwellian. The challenge arises from the nonlocal quadratic interaction term defined in the unbounded domain of velocity space. Analyzing this term on an unbounded domain requires the inclusion of a truncation function, which demands delicate analysis techniques. As a generalization of this analysis, we also provide proof of the asymptotic preserving property when using micro-macro decomposition-based neural networks.
△ Less
Submitted 11 July, 2024;
originally announced July 2024.
-
Cryptography-Based Privacy-Preserving Method for Distributed Optimization over Time-Varying Directed Graphs with Enhanced Efficiency
Authors:
Bing Liu,
Furan Xie,
Li Chai
Abstract:
In this paper, we study the privacy-preserving distributed optimization problem, aiming to prevent attackers from stealing the private information of agents. For this purpose, we propose a novel privacy-preserving algorithm based on the Advanced Encryption Standard (AES), which is both secure and computationally efficient. By appropriately constructing the underlying weight matrices, our algorithm…
▽ More
In this paper, we study the privacy-preserving distributed optimization problem, aiming to prevent attackers from stealing the private information of agents. For this purpose, we propose a novel privacy-preserving algorithm based on the Advanced Encryption Standard (AES), which is both secure and computationally efficient. By appropriately constructing the underlying weight matrices, our algorithm can be applied to time-varying directed networks. We show that the proposed algorithm can protect an agent's privacy if the agent has at least one legitimate neighbor at the initial iteration. Under the assumption that the objective function is strongly convex and Lipschitz smooth, we rigorously prove that the proposed algorithm has a linear convergence rate. Finally, the effectiveness of the proposed algorithm is demonstrated by numerical simulations of the canonical sensor fusion problem.
△ Less
Submitted 14 May, 2024;
originally announced May 2024.
-
Frozen Gaussian approximation for the fractional Schrödinger equation
Authors:
Lihui Chai,
Hengzhun Chen,
Xu Yang
Abstract:
We develop the frozen Gaussian approximation (FGA) for the fractional Schrödinger equation in the semi-classical regime, where the solution is highly oscillatory when the scaled Planck constant $\varepsilon$ is small. This method approximates the solution to the Schrödinger equation by an integral representation based on asymptotic analysis and provides a highly efficient computational method for…
▽ More
We develop the frozen Gaussian approximation (FGA) for the fractional Schrödinger equation in the semi-classical regime, where the solution is highly oscillatory when the scaled Planck constant $\varepsilon$ is small. This method approximates the solution to the Schrödinger equation by an integral representation based on asymptotic analysis and provides a highly efficient computational method for high-frequency wave function evolution. In particular, we revise the standard FGA formula to address the singularities arising in the higher-order derivatives of coefficients of the associated Hamiltonian flow that are second-order continuously differentiable or smooth in conventional FGA analysis. We then establish its convergence to the true solution. Additionally, we provide some numerical examples to verify the accuracy and convergence behavior of the frozen Gaussian approximation method.
△ Less
Submitted 27 March, 2024;
originally announced March 2024.
-
Distributed Stochastic Bilevel Optimization: Improved Complexity and Heterogeneity Analysis
Authors:
Youcheng Niu,
Jinming Xu,
Ying Sun,
Yan Huang,
Li Chai
Abstract:
This paper consider solving a class of nonconvex-strongly-convex distributed stochastic bilevel optimization (DSBO) problems with personalized inner-level objectives. Most existing algorithms require computational loops for hypergradient estimation, leading to computational inefficiency. Moreover, the impact of data heterogeneity on convergence in bilevel problems is not explicitly characterized y…
▽ More
This paper consider solving a class of nonconvex-strongly-convex distributed stochastic bilevel optimization (DSBO) problems with personalized inner-level objectives. Most existing algorithms require computational loops for hypergradient estimation, leading to computational inefficiency. Moreover, the impact of data heterogeneity on convergence in bilevel problems is not explicitly characterized yet. To address these issues, we propose LoPA, a loopless personalized distributed algorithm that leverages a tracking mechanism for iterative approximation of inner-level solutions and Hessian-inverse matrices without relying on extra computation loops. Our theoretical analysis explicitly characterizes the heterogeneity across nodes (denoted by $b$), and establishes a sublinear rate of $\mathcal{O}( {\frac{1}{{{{\left( {1 - ρ} \right)}}K}} \!+ \!\frac{{(\frac{b}{\sqrt{m}})^{\frac{2}{3}} }}{{\left( {1 - ρ} \right)^{\frac{2}{3}} K^{\frac{2}{3}} }} \!+ \!\frac{1}{\sqrt{ K }}( {σ_{\operatorname{p} }} + \frac{1}{\sqrt{m}}{σ_{\operatorname{c} }} ) } )$ without the boundedness of local hypergradients, where ${σ_{\operatorname{p} }}$ and ${σ_{\operatorname{c} }}$ represent the gradient sampling variances associated with the inner- and outer-level variables, respectively. We also integrate LoPA with a gradient tracking scheme to eliminate the impact of data heterogeneity, yielding an improved rate of ${\mathcal{O}}(\frac{1}{ (1-ρ)^2K } \!+\! \frac{1}{\sqrt{K}}( σ_{\rm{p}} \!+\! \frac{1}{\sqrt{m}}σ_{\rm{c}} ) )$. The computational complexity of LoPA is of ${\mathcal{O}}({ε^{-2}})$ to an $ε$-stationary point, matching the communication complexity due to the loopless structure, which outperforms existing counterparts for DSBO. Numerical experiments validate the effectiveness of the proposed algorithm.
△ Less
Submitted 7 April, 2025; v1 submitted 22 December, 2023;
originally announced December 2023.
-
Accelerating the Convergence Rate of Consensus for Second-Order Multi-Agent Systems by Memory Information
Authors:
Jiahao Dai,
Jing-Wen Yi,
Li Chai
Abstract:
This paper utilizes the agent's memory in accelerated consensus for second-order multi-agent systems (MASs). In the case of one-tap memory, explicit formulas for the optimal consensus convergence rate and control parameters are derived by applying the Jury stability criterion. It is proved that the optimal consensus convergence rate with one-tap memory is faster than that without memory. In the ca…
▽ More
This paper utilizes the agent's memory in accelerated consensus for second-order multi-agent systems (MASs). In the case of one-tap memory, explicit formulas for the optimal consensus convergence rate and control parameters are derived by applying the Jury stability criterion. It is proved that the optimal consensus convergence rate with one-tap memory is faster than that without memory. In the case of M-tap memory, an iterative algorithm is given to derive the control parameters to accelerate the convergence rate. Moreover, the accelerated consensus with one-tap memory is extended to the formation control, and the control parameters to achieve the fastest formation are obtained. Numerical examples further illustrate the theoretical results.
△ Less
Submitted 24 March, 2023;
originally announced March 2023.
-
Frozen Gaussian Sampling for Scalar Wave Equations
Authors:
Lihui Chai,
Ye Feng,
Zhennan Zhou
Abstract:
In this article, we introduce the frozen Gaussian sampling (FGS) algorithm to solve the scalar wave equation in the high-frequency regime. The FGS algorithm is a Monte Carlo sampling strategy based on the frozen Gaussian approximation, which greatly reduces the computation workload in the wave propagation and reconstruction. In this work, we propose feasible and detailed procedures to implement th…
▽ More
In this article, we introduce the frozen Gaussian sampling (FGS) algorithm to solve the scalar wave equation in the high-frequency regime. The FGS algorithm is a Monte Carlo sampling strategy based on the frozen Gaussian approximation, which greatly reduces the computation workload in the wave propagation and reconstruction. In this work, we propose feasible and detailed procedures to implement the FGS algorithm to approximate scalar wave equations with Gaussian initial conditions and WKB initial conditions respectively. For both initial data cases, we rigorously analyze the error of applying this algorithm to wave equations of dimensionality $d \geq 3$. In Gaussian initial data cases, we prove that the sampling error due to the Monte Carlo method is independent of the typical wave number. We also derive a quantitative bound of the sampling error in WKB initial data cases. Finally, we validate the performance of the FGS and the theoretical estimates about the sampling error through various numerical examples, which include using the FGS to solve wave equations with both Gaussian and WKB initial data of dimensionality $d = 1, 2$, and $3$.
△ Less
Submitted 2 September, 2023; v1 submitted 9 November, 2022;
originally announced November 2022.
-
Fast consensus of high-order multi-agent systems
Authors:
Jiahao Dai,
Jing-Wen Yi,
Li Chai
Abstract:
In this paper, the fast consensus problem of high-order multi-agent systems under undirected topologies is considered. The direct link between the consensus convergence rate and the control gains is established. An accelerated consensus algorithm based on gradient descent is proposed to optimize the convergence rate. By applying the Routh-Hurwitz stability criterion, the lower bound on the converg…
▽ More
In this paper, the fast consensus problem of high-order multi-agent systems under undirected topologies is considered. The direct link between the consensus convergence rate and the control gains is established. An accelerated consensus algorithm based on gradient descent is proposed to optimize the convergence rate. By applying the Routh-Hurwitz stability criterion, the lower bound on the convergence rate is derived, and explicit control gains are derived as the necessary condition to achieve the optimal convergence rate. Moreover, a protocol with time-varying control gains is designed to achieve the finite-time consensus. Explicit formulas for the time-varying control gains and the final consensus state are given. Numerical examples and simulation results are presented to illustrate the obtained theoretical results.
△ Less
Submitted 16 May, 2022;
originally announced May 2022.
-
Optimal Memory Scheme for Accelerated Consensus Over Multi-Agent Networks
Authors:
Jiahao Dai,
Jing-Wen Yi,
Li Chai
Abstract:
The consensus over multi-agent networks can be accelerated by introducing agent's memory to the control protocol. In this paper, a more general protocol with the node memory and the state deviation memory is designed. We aim to provide the optimal memory scheme to accelerate consensus. The contributions of this paper are three: (i) For the one-tap memory scheme, we demonstrate that the state devia…
▽ More
The consensus over multi-agent networks can be accelerated by introducing agent's memory to the control protocol. In this paper, a more general protocol with the node memory and the state deviation memory is designed. We aim to provide the optimal memory scheme to accelerate consensus. The contributions of this paper are three: (i) For the one-tap memory scheme, we demonstrate that the state deviation memory is useless for the optimal convergence. (ii) In the worst case, we prove that it is a vain to add any tap of the state deviation memory, and the one-tap node memory is sufficient to achieve the optimal convergence. (iii) We show that the two-tap state deviation memory is effective on some special networks, such as star networks. Numerical examples are listed to illustrate the validity and correctness of the obtained results.
△ Less
Submitted 13 December, 2021;
originally announced December 2021.
-
Convergence Rate of Accelerated Average Consensus with Local Node Memory: Optimization and Analytic Solutions
Authors:
Jing-Wen Yi,
Li Chai,
Jingxin Zhang
Abstract:
Previous researches have shown that adding local memory can accelerate the consensus. It is natural to ask questions like what is the fastest rate achievable by the $M$-tap memory acceleration, and what are the corresponding control parameters. This paper introduces a set of effective and previously unused techniques to analyze the convergence rate of accelerated consensus with $M$-tap memory of l…
▽ More
Previous researches have shown that adding local memory can accelerate the consensus. It is natural to ask questions like what is the fastest rate achievable by the $M$-tap memory acceleration, and what are the corresponding control parameters. This paper introduces a set of effective and previously unused techniques to analyze the convergence rate of accelerated consensus with $M$-tap memory of local nodes and to design the control protocols. These effective techniques, including the Kharitonov stability theorem, the Routh stability criterion and the robust stability margin, have led to the following new results: 1) the direct link between the convergence rate and the control parameters; 2) explicit formulas of the optimal convergence rate and the corresponding optimal control parameters for $M \leq 2$ on a given graph; 3) the optimal worst-case convergence rate and the corresponding optimal control parameters for the memory $M \geq 1$ on a set of uncertain graphs. We show that the acceleration with the memory $M = 1$ provides the optimal convergence rate in the sense of worst-case performance. Several numerical examples are given to demonstrate the validity and performance of the theoretical results.
△ Less
Submitted 10 December, 2021; v1 submitted 18 October, 2021;
originally announced October 2021.
-
Seismic Tomography with Random Batch Gradient Reconstruction
Authors:
Yixiao Hu,
Lihui Chai,
Zhongyi Huang,
Xu Yang
Abstract:
Seismic tomography solves high-dimensional optimization problems to image subsurface structures of Earth. In this paper, we propose to use random batch methods to construct the gradient used for iterations in seismic tomography. Specifically, we use the frozen Gaussian approximation to compute seismic wave propagation, and then construct stochastic gradients by random batch methods. The method inh…
▽ More
Seismic tomography solves high-dimensional optimization problems to image subsurface structures of Earth. In this paper, we propose to use random batch methods to construct the gradient used for iterations in seismic tomography. Specifically, we use the frozen Gaussian approximation to compute seismic wave propagation, and then construct stochastic gradients by random batch methods. The method inherits the spirit of stochastic gradient descent methods for solving high-dimensional optimization problems. The proposed idea is general in the sense that it does not rely on the usage of the frozen Gaussian approximation, and one can replace it with any other efficient wave propagation solvers, e.g., Gaussian beam methods and spectral element methods. We prove the convergence of the random batch method in the mean-square sense, and show the numerical performance of the proposed method by two-dimensional and three-dimensional examples of wave-equation-based travel-time inversion and full-waveform inversion, respectively. As a byproduct, we also prove the convergence of the accelerated full-waveform inversion using dynamic mini-batches and spectral element methods.
△ Less
Submitted 11 February, 2023; v1 submitted 12 October, 2021;
originally announced October 2021.
-
Opinion Dynamics Models with Memory in Coopetitive Social Networks: Analysis, Application and Simulation
Authors:
Qingsong Liu,
Li Chai
Abstract:
In some social networks, the opinion forming is based on its own and neighbors' (initial) opinions, whereas the evolution of the individual opinions is also influenced by the individual's past opinions in the real world. Unlike existing social network models, in this paper, a novel model of opinion dynamics is proposed, which describes the evolution of the individuals' opinions not only depends on…
▽ More
In some social networks, the opinion forming is based on its own and neighbors' (initial) opinions, whereas the evolution of the individual opinions is also influenced by the individual's past opinions in the real world. Unlike existing social network models, in this paper, a novel model of opinion dynamics is proposed, which describes the evolution of the individuals' opinions not only depends on its own and neighbors' current opinions, but also depends on past opinions. Memory and memoryless communication rules are simultaneously established for the proposed opinion dynamics model. Sufficient and/or necessary conditions for the equal polarization, consensus and neutralizability of the opinions are respectively presented in terms of the network topological structure and the spectral analysis. We apply our model to simulate Kahneman's seminal experiments on choices in risky and riskless contexts, which fits in with the experiment results. Simulation analysis shows that the memory capacity of the individuals is inversely proportional to the speeds of the ultimate opinions formational.
△ Less
Submitted 5 August, 2021;
originally announced August 2021.
-
A multi-band semiclassical model for surface hopping quantum dynamics
Authors:
Lihui Chai,
Shi Jin,
Qin Li,
Omar Morandi
Abstract:
In the paper we derive a semiclassical model for surface hopping allowing quantum dynamical non-adiabatic transition between different potential energy surfaces in which cases the classical Born-Oppenheimer approximation breaks down. The model is derived using the Wigner transform and Weyl quantization, and the central idea is to evolve the entire Wigner matrix rather than just the diagonal entrie…
▽ More
In the paper we derive a semiclassical model for surface hopping allowing quantum dynamical non-adiabatic transition between different potential energy surfaces in which cases the classical Born-Oppenheimer approximation breaks down. The model is derived using the Wigner transform and Weyl quantization, and the central idea is to evolve the entire Wigner matrix rather than just the diagonal entries as was done previously in the adiabatic case. The off-diagonal entries of the Wigner matrix suitably describe the non-adiabatic transition, such as the Berry connection, for avoided crossings. We study the numerical approximation issues of the model, and then conduct numerical experiments to validate the model.
△ Less
Submitted 4 May, 2014;
originally announced May 2014.
