Title:Variance-based stochastic extragradient methods with linear search for stochastic variational inequalities

Abstract:We propose stochastic extragradient methods for stochastic variational inequalities with a linear search requiring only pseudo-monotonicity of the operator and no knowledge of the Lipschitz constant $L$. We provide convergence and complexity analysis, allowing for an unbounded feasible set, unbounded operator, non-uniform variance of the oracle and we do not require any regularization. We also prove the generated sequence is bounded in L$^p$. Alongside the stochastic approximation procedure, we iteratively reduce the variance of the stochastic error. Our methods cope with stepsizes bounded away from zero and attain the near-optimal oracle complexity $O(\log_{1/\theta}L)\cdot\epsilon^{-2}\cdot[\ln(\epsilon^{-1})]^{1+b}$ and an accelerated rate $O(1/K)$ in terms of the mean (quadratic) natural residual and the mean D-gap function, where $K$ is the number of iterations required for a given tolerance $\epsilon>0$ for arbitrary $\theta\in(0,1)$ and $b>0$. Explicit estimates for the convergence rate, oracle complexity and the $p$-moments are given depending on problem parameters and the distance of initial iterates to the solution set.