Showing 1–2 of 2 results for author: Tung, T T

On the boundedness of the sequence generated by minibatch stochastic gradient descent

Authors: Heinz H. Bauschke, Tran Thanh Tung

Abstract: Stochastic Gradient Descent (SGD) with Polyak's stepsize has recently gained renewed attention in stochastic optimization. Recently, Orvieto, Lacoste-Julien, and Loizou introduced a decreasing variant of Polyak's stepsize, where convergence relies on a boundedness assumption of the iterates. They established that this assumption holds under strong convexity. In this paper, we extend their result b… ▽ More Stochastic Gradient Descent (SGD) with Polyak's stepsize has recently gained renewed attention in stochastic optimization. Recently, Orvieto, Lacoste-Julien, and Loizou introduced a decreasing variant of Polyak's stepsize, where convergence relies on a boundedness assumption of the iterates. They established that this assumption holds under strong convexity. In this paper, we extend their result by proving that boundedness also holds for a broader class of objective functions, including coercive functions. We also present a case in which boundedness may or may not hold. △ Less

Submitted 29 June, 2025; originally announced June 2025.

MSC Class: Primary 90C15; 90C25; 65K05; Secondary 68T07; 68W20; 68W40

On a result by Meshulam

Authors: Heinz H. Bauschke, Tran Thanh Tung

Abstract: In 1996, Meshulam proved that every sequence generated by applying projections onto affine subspaces, drawn from a finite collection in Euclidean space, must be bounded. In this paper, we extend his result not only from affine subspaces to convex polyhedral subsets, but also from Euclidean to general Hilbert space. Various examples are provided to illustrate the sharpness of the results. In 1996, Meshulam proved that every sequence generated by applying projections onto affine subspaces, drawn from a finite collection in Euclidean space, must be bounded. In this paper, we extend his result not only from affine subspaces to convex polyhedral subsets, but also from Euclidean to general Hilbert space. Various examples are provided to illustrate the sharpness of the results. △ Less

Submitted 27 June, 2025; originally announced June 2025.

MSC Class: 47H09; 65K05; 90C25 (Primary) 52A37; 52B55 (Secondary)