Title:Optimal Distributed Optimization on Slowly Time-Varying Graphs

Abstract:We study the convergence rate of first-order optimization algorithms when the objective function can change from one iteration to another, but its minimizer and optimal value remain the same. This problem is motivated by recent developments in optimal distributed optimization algorithms over networks where computational nodes or agents can experience network malfunctions such as a loss of connection between two nodes. We show an explicit and non-asymptotic linear convergence of the distributed versions of the gradient descent and Nesterov's fast gradient method on strongly convex and smooth objective functions when the network of nodes has a finite number of changes (we will call this network \textit{slowly} time-varying). Moreover, we show that Nesterov method reaches the optimal iteration complexity of $\Omega(\sqrt{\kappa\cdot\chi(W)}\log\frac{1}{\eps})$ for decentralized algorithms, where $\kappa$ and $\chi(W)$ are condition numbers from the objective function and communication graphs respectively.