Showing 1–10 of 10 results for author: Lall, S

Modeling Buffer Occupancy in bittide Systems

Authors: Sanjay Lall, Tammo Spalink

Abstract: The bittide mechanism enables logically synchronous computation across distributed systems by leveraging the continuous frame transmission inherent to wired networks such as Ethernet. Instead of relying on a global clock, bittide uses a decentralized control system to adjust local clock frequencies, ensuring all nodes operate with a consistent notion of time by utilizing elastic buffers at each no… ▽ More The bittide mechanism enables logically synchronous computation across distributed systems by leveraging the continuous frame transmission inherent to wired networks such as Ethernet. Instead of relying on a global clock, bittide uses a decentralized control system to adjust local clock frequencies, ensuring all nodes operate with a consistent notion of time by utilizing elastic buffers at each node to absorb frequency variations. This paper presents an analysis of the steady-state occupancy of these elastic buffers, a critical factor influencing system latency. Using a fluid model of the bittide system, we prove that buffer occupancy converges and derive an explicit formula for the steady-state value in terms of system parameters, including network topology, physical latencies, and controller gains. This analysis provides valuable insights for optimizing buffer sizes and minimizing latency in bittide-based distributed systems. △ Less

Submitted 7 October, 2024; originally announced October 2024.

Markov Decision Processes with Noisy State Observation

Authors: Amirhossein Afsharrad, Sanjay Lall

Abstract: This paper addresses the challenge of a particular class of noisy state observations in Markov Decision Processes (MDPs), a common issue in various real-world applications. We focus on modeling this uncertainty through a confusion matrix that captures the probabilities of misidentifying the true state. Our primary goal is to estimate the inherent measurement noise, and to this end, we propose two… ▽ More This paper addresses the challenge of a particular class of noisy state observations in Markov Decision Processes (MDPs), a common issue in various real-world applications. We focus on modeling this uncertainty through a confusion matrix that captures the probabilities of misidentifying the true state. Our primary goal is to estimate the inherent measurement noise, and to this end, we propose two novel algorithmic approaches. The first, the method of second-order repetitive actions, is designed for efficient noise estimation within a finite time window, providing identifiable conditions for system analysis. The second approach comprises a family of Bayesian algorithms, which we thoroughly analyze and compare in terms of performance and limitations. We substantiate our theoretical findings with simulations, demonstrating the effectiveness of our methods in different scenarios, particularly highlighting their behavior in environments with varying stationary distributions. Our work advances the understanding of reinforcement learning in noisy environments, offering robust techniques for more accurate state estimation in MDPs. △ Less

Submitted 13 December, 2023; originally announced December 2023.

Convex Methods for Constrained Linear Bandits

Authors: Amirhossein Afsharrad, Ahmadreza Moradipari, Sanjay Lall

Abstract: Recently, bandit optimization has received significant attention in real-world safety-critical systems that involve repeated interactions with humans. While there exist various algorithms with performance guarantees in the literature, practical implementation of the algorithms has not received as much attention. This work presents a comprehensive study on the computational aspects of safe bandit a… ▽ More Recently, bandit optimization has received significant attention in real-world safety-critical systems that involve repeated interactions with humans. While there exist various algorithms with performance guarantees in the literature, practical implementation of the algorithms has not received as much attention. This work presents a comprehensive study on the computational aspects of safe bandit algorithms, specifically safe linear bandits, by introducing a framework that leverages convex programming tools to create computationally efficient policies. In particular, we first characterize the properties of the optimal policy for safe linear bandit problem and then propose an end-to-end pipeline of safe linear bandit algorithms that only involves solving convex problems. We also numerically evaluate the performance of our proposed methods. △ Less

Submitted 9 November, 2023; v1 submitted 7 November, 2023; originally announced November 2023.

On Buffer Centering for Bittide Synchronization

Authors: Sanjay Lall, Calin Cascaval, Martin Izzard, Tammo Spalink

Abstract: We discuss distributed reframing control of bittide systems. In a bittide system, multiple processors synchronize by monitoring communication over the network. The processors remain in logical synchrony by controlling the timing of frame transmissions. The protocol for doing this relies upon an underlying dynamic control system, where each node makes only local observations and performs no direct… ▽ More We discuss distributed reframing control of bittide systems. In a bittide system, multiple processors synchronize by monitoring communication over the network. The processors remain in logical synchrony by controlling the timing of frame transmissions. The protocol for doing this relies upon an underlying dynamic control system, where each node makes only local observations and performs no direct coordination with other nodes. In this paper we develop a control algorithm based on the idea of reset control, which allows all nodes to maintain small buffer offsets while also requiring very little state information at each node. We demonstrate that with reframing, we can achieve separate control of frequency and phase, allowing both the frequencies to be syntonized and the buffers to be moved the desired points, rather than combining their control via a proportional-integral controller. This offers the potential for simplified boot processes and failure handling. △ Less

Submitted 20 March, 2023; originally announced March 2023.

Probabilistic Modeling Using Tree Linear Cascades

Authors: Nicholas C. Landolfi, Sanjay Lall

Abstract: We introduce tree linear cascades, a class of linear structural equation models for which the error variables are uncorrelated but need not be Gaussian nor independent. We show that, in spite of this weak assumption, the tree structure of this class of models is identifiable. In a similar vein, we introduce a constrained regression problem for fitting a tree-structured linear structural equation m… ▽ More We introduce tree linear cascades, a class of linear structural equation models for which the error variables are uncorrelated but need not be Gaussian nor independent. We show that, in spite of this weak assumption, the tree structure of this class of models is identifiable. In a similar vein, we introduce a constrained regression problem for fitting a tree-structured linear structural equation model and solve the problem analytically. We connect these results to the classical Chow-Liu approach for Gaussian graphical models. We conclude by giving an empirical-risk form of the regression and illustrating the computationally attractive implications of our theoretical results on a basic example involving stock prices. △ Less

Submitted 15 February, 2022; originally announced February 2022.

Comments: long form of an article to appear in the proceedings of the 2022 American Control Conference (ACC 2022). 8 pages, 1 figure; includes an appendix which the conference version omits

Resistance Distance and Control Performance for bittide Synchronization

Authors: Sanjay Lall, Calin Cascaval, Martin Izzard, Tammo Spalink

Abstract: We discuss control of bittide distributed systems, which are designed to provide logical synchronization between networked machines by observing data flow rates between adjacent systems at the physical network layer and controlling local reference clock frequencies. We analyze the performance of approximate proportional-integral control of the synchronization mechanism and develop a simple continu… ▽ More We discuss control of bittide distributed systems, which are designed to provide logical synchronization between networked machines by observing data flow rates between adjacent systems at the physical network layer and controlling local reference clock frequencies. We analyze the performance of approximate proportional-integral control of the synchronization mechanism and develop a simple continuous-time model to show the resulting dynamics are stable for any positive choice of gains. We then construct explicit formulae to show that closed-loop performance measured using the L2 norm is a product of two terms, one depending only on resistance distances in the graph, and the other depending only on controller gains. △ Less

Submitted 31 March, 2022; v1 submitted 9 November, 2021; originally announced November 2021.

Modeling and Control of bittide Synchronization

Authors: Sanjay Lall, Calin Cascaval, Martin Izzard, Tammo Spalink

Abstract: Distributed system applications rely on a fine-grain common sense of time. Existing systems maintain the common sense of time by keeping each independent machine as close as possible to wall-clock time through a combination of software protocols like NTP and GPS signals and/or precision references like atomic clocks. This approach is expensive and has tolerance limitations that require protocols t… ▽ More Distributed system applications rely on a fine-grain common sense of time. Existing systems maintain the common sense of time by keeping each independent machine as close as possible to wall-clock time through a combination of software protocols like NTP and GPS signals and/or precision references like atomic clocks. This approach is expensive and has tolerance limitations that require protocols to deal with asynchrony and its performance consequences. Moreover, at data-center scale it is impractical to distribute a physical clock as is done on a chip or printed circuit board. In this paper we introduce a distributed system design that removes the need for physical clock distribution or mechanisms for maintaining close alignment to wall-clock time, and instead provides applications with a perfectly synchronized logical clock. We discuss the abstract frame model (AFM), a mathematical model that underpins the system synchronization. The model is based on the rate of communication between nodes in a topology without requiring a global clock. We show that there are families of controllers that satisfy the properties required for existence and uniqueness of solutions to the AFM, and give examples. △ Less

Submitted 31 March, 2022; v1 submitted 28 September, 2021; originally announced September 2021.

Comments: 8 pages, 2 figures

An Algebraic Approach to the Control of Decentralized Systems

Authors: Laurent Lessard, Sanjay Lall

Abstract: Optimal decentralized controller design is notoriously difficult, but recent research has identified large subclasses of such problems that may be convexified and thus are amenable to solution via efficient numerical methods. One recently discovered sufficient condition for convexity is quadratic invariance (QI). Despite the simple algebraic characterization of QI, which relates the plant and cont… ▽ More Optimal decentralized controller design is notoriously difficult, but recent research has identified large subclasses of such problems that may be convexified and thus are amenable to solution via efficient numerical methods. One recently discovered sufficient condition for convexity is quadratic invariance (QI). Despite the simple algebraic characterization of QI, which relates the plant and controller maps, proving convexity of the set of achievable closed-loop maps requires tools from functional analysis. In this work, we present a new formulation of quadratic invariance that is purely algebraic. While our results are similar in flavor to those from traditional QI theory, they do not follow from that body of work. Furthermore, they are applicable to new types of systems that are difficult to treat using functional analysis. Examples discussed include rational transfer matrices, systems with delays, and multidimensional systems. △ Less

Submitted 21 November, 2014; v1 submitted 20 September, 2013; originally announced September 2013.

Journal ref: IEEE Transactions on Control of Network Systems, Vol.1(4), pp.1-10, Dec. 2014

Convexity of Decentralized Controller Synthesis

Authors: Laurent Lessard, Sanjay Lall

Abstract: In decentralized control problems, a standard approach is to specify the set of allowable decentralized controllers as a closed subspace of linear operators. This then induces a corresponding set of Youla parameters. Previous work has shown that quadratic invariance of the controller set implies that the set of Youla parameters is convex. In this paper, we prove the converse. We thereby show that… ▽ More In decentralized control problems, a standard approach is to specify the set of allowable decentralized controllers as a closed subspace of linear operators. This then induces a corresponding set of Youla parameters. Previous work has shown that quadratic invariance of the controller set implies that the set of Youla parameters is convex. In this paper, we prove the converse. We thereby show that the only decentralized control problems for which the set of Youla parameters is convex are those which are quadratically invariant. We further show that under additional assumptions, quadratic invariance is necessary and sufficient for the set of achievable closed-loop maps to be convex. We give two versions of our results. The first applies to bounded linear operators on a Banach space and the second applies to (possibly unstable) causal LTI systems in discrete or continuous time. △ Less

Submitted 27 January, 2014; v1 submitted 24 May, 2013; originally announced May 2013.

Optimal Control of Two-Player Systems with Output Feedback

Authors: Laurent Lessard, Sanjay Lall

Abstract: In this article, we consider a fundamental decentralized optimal control problem, which we call the two-player problem. Two subsystems are interconnected in a nested information pattern, and output feedback controllers must be designed for each subsystem. Several special cases of this architecture have previously been solved, such as the state-feedback case or the case where the dynamics of both s… ▽ More In this article, we consider a fundamental decentralized optimal control problem, which we call the two-player problem. Two subsystems are interconnected in a nested information pattern, and output feedback controllers must be designed for each subsystem. Several special cases of this architecture have previously been solved, such as the state-feedback case or the case where the dynamics of both systems are decoupled. In this paper, we present a detailed solution to the general case. The structure of the optimal decentralized controller is reminiscent of that of the optimal centralized controller; each player must estimate the state of the system given their available information and apply static control policies to these estimates to compute the optimal controller. The previously solved cases benefit from a separation between estimation and control which allows one to compute the control and estimation gains separately. This feature is not present in general, and some of the gains must be solved for simultaneously. We show that computing the required coupled estimation and control gains amounts to solving a small system of linear equations. △ Less

Submitted 8 July, 2015; v1 submitted 14 March, 2013; originally announced March 2013.