Showing 1–50 of 55 results for author: Tron, R

Real-Time Navigation for Autonomous Aerial Vehicles Using Video

Authors: Khizar Anjum, Parul Pandey, Vidyasagar Sadhu, Roberto Tron, Dario Pompili

Abstract: Most applications in autonomous navigation using mounted cameras rely on the construction and processing of geometric 3D point clouds, which is an expensive process. However, there is another simpler way to make a space navigable quickly: to use semantic information (e.g., traffic signs) to guide the agent. However, detecting and acting on semantic information involves Computer Vision~(CV) algorit… ▽ More Most applications in autonomous navigation using mounted cameras rely on the construction and processing of geometric 3D point clouds, which is an expensive process. However, there is another simpler way to make a space navigable quickly: to use semantic information (e.g., traffic signs) to guide the agent. However, detecting and acting on semantic information involves Computer Vision~(CV) algorithms such as object detection, which themselves are demanding for agents such as aerial drones with limited onboard resources. To solve this problem, we introduce a novel Markov Decision Process~(MDP) framework to reduce the workload of these CV approaches. We apply our proposed framework to both feature-based and neural-network-based object-detection tasks, using open-loop and closed-loop simulations as well as hardware-in-the-loop emulations. These holistic tests show significant benefits in energy consumption and speed with only a limited loss in accuracy compared to models based on static features and neural networks. △ Less

Submitted 31 March, 2025; originally announced April 2025.

Comments: Submitted to Journal of Real-Time Image Processing

Distributed Online Task Assignment via Inexact ADMM for unplanned online tasks and its Applications to Security

Authors: Ziqi Yang, Roberto Tron

Abstract: In multi-robot system (MRS) applications, efficient task assignment is essential not only for coordinating agents and ensuring mission success but also for maintaining overall system security. In this work, we first propose an optimization-based distributed task assignment algorithm that dynamically assigns mandatory security-critical tasks and optional tasks among teams. Leveraging an inexact Alt… ▽ More In multi-robot system (MRS) applications, efficient task assignment is essential not only for coordinating agents and ensuring mission success but also for maintaining overall system security. In this work, we first propose an optimization-based distributed task assignment algorithm that dynamically assigns mandatory security-critical tasks and optional tasks among teams. Leveraging an inexact Alternating Direction Method of Multipliers (ADMM)-based approach, we decompose the task assignment problem into separable and non-separable subproblems. The non-separable subproblems are transformed into an inexact ADMM update by projected gradient descent, which can be performed through several communication steps within the team. In the second part of this paper, we formulate a comprehensive framework that enables MRS under plan-deviation attacks to handle online tasks without compromising security. The process begins with a security analysis that determines whether an online task can be executed securely by a robot and, if so, the required time and location for the robot to rejoin the team. Next, the proposed task assignment algorithm is used to allocate security-related tasks and verified online tasks. Finally, task fulfillment is managed using a Control Lyapunov Function (CLF)-based controller, while security enforcement is ensured through a Control Barrier Function (CBF)-based security filter. Through simulations, we demonstrate that the proposed framework allows MRS to effectively respond to unplanned online tasks while maintaining security guarantees. △ Less

Submitted 26 February, 2025; originally announced February 2025.

Comments: Submitted to IEEE TCNS

Adaptive Bi-Level Multi-Robot Task Allocation and Learning under Uncertainty with Temporal Logic Constraints

Authors: Xiaoshan Lin, Roberto Tron

Abstract: This work addresses the problem of multi-robot coordination under unknown robot transition models, ensuring that tasks specified by Time Window Temporal Logic are satisfied with user-defined probability thresholds. We present a bi-level framework that integrates (i) high-level task allocation, where tasks are assigned based on the robots' estimated task completion probabilities and expected reward… ▽ More This work addresses the problem of multi-robot coordination under unknown robot transition models, ensuring that tasks specified by Time Window Temporal Logic are satisfied with user-defined probability thresholds. We present a bi-level framework that integrates (i) high-level task allocation, where tasks are assigned based on the robots' estimated task completion probabilities and expected rewards, and (ii) low-level distributed policy learning and execution, where robots independently optimize auxiliary rewards while fulfilling their assigned tasks. To handle uncertainty in robot dynamics, our approach leverages real-time task execution data to iteratively refine expected task completion probabilities and rewards, enabling adaptive task allocation without explicit robot transition models. We theoretically validate the proposed algorithm, demonstrating that the task assignments meet the desired probability thresholds with high confidence. Finally, we demonstrate the effectiveness of our framework through comprehensive simulations. △ Less

Submitted 14 February, 2025; originally announced February 2025.

Comments: Accepted as a full paper at AAMAS 2025

Accelerating Proximal Policy Optimization Learning Using Task Prediction for Solving Environments with Delayed Rewards

Authors: Ahmad Ahmad, Mehdi Kermanshah, Kevin Leahy, Zachary Serlin, Ho Chit Siu, Makai Mann, Cristian-Ioan Vasile, Roberto Tron, Calin Belta

Abstract: In this paper, we tackle the challenging problem of delayed rewards in reinforcement learning (RL). While Proximal Policy Optimization (PPO) has emerged as a leading Policy Gradient method, its performance can degrade under delayed rewards. We introduce two key enhancements to PPO: a hybrid policy architecture that combines an offline policy (trained on expert demonstrations) with an online PPO po… ▽ More In this paper, we tackle the challenging problem of delayed rewards in reinforcement learning (RL). While Proximal Policy Optimization (PPO) has emerged as a leading Policy Gradient method, its performance can degrade under delayed rewards. We introduce two key enhancements to PPO: a hybrid policy architecture that combines an offline policy (trained on expert demonstrations) with an online PPO policy, and a reward shaping mechanism using Time Window Temporal Logic (TWTL). The hybrid architecture leverages offline data throughout training while maintaining PPO's theoretical guarantees. Building on the monotonic improvement framework of Trust Region Policy Optimization (TRPO), we prove that our approach ensures improvement over both the offline policy and previous iterations, with a bounded performance gap of $(2ςγα^2)/(1-γ)^2$, where $α$ is the mixing parameter, $γ$ is the discount factor, and $ς$ bounds the expected advantage. Additionally, we prove that our TWTL-based reward shaping preserves the optimal policy of the original problem. TWTL enables formal translation of temporal objectives into immediate feedback signals that guide learning. We demonstrate the effectiveness of our approach through extensive experiments on an inverted pendulum and a lunar lander environments, showing improvements in both learning speed and final performance compared to standard PPO and offline-only approaches. △ Less

Submitted 4 December, 2024; v1 submitted 26 November, 2024; originally announced November 2024.

BoxMap: Efficient Structural Mapping and Navigation

Authors: Zili Wang, Christopher Allum, Sean B. Andersson, Roberto Tron

Abstract: While humans can successfully navigate using abstractions, ignoring details that are irrelevant to the task at hand, most existing robotic applications require the maintenance of a detailed environment representation which consumes a significant amount of sensing, computing, and storage. These issues are particularly important in a resource-constrained setting with limited power budget. Deep learn… ▽ More While humans can successfully navigate using abstractions, ignoring details that are irrelevant to the task at hand, most existing robotic applications require the maintenance of a detailed environment representation which consumes a significant amount of sensing, computing, and storage. These issues are particularly important in a resource-constrained setting with limited power budget. Deep learning methods can learn from prior experience to abstract knowledge of unknown environments, and use it to execute tasks (e.g., frontier exploration, object search, or scene understanding) more efficiently. We propose BoxMap, a Detection-Transformer-based architecture that takes advantage of the structure of the sensed partial environment to update a topological graph of the environment as a set of semantic entities (e.g. rooms and doors) and their relations (e.g. connectivity). These predictions from low-level measurements can then be leveraged to achieve high-level goals with lower computational costs than methods based on detailed representations. As an example application, we consider a robot equipped with a 2-D laser scanner tasked with exploring a residential building. Our BoxMap representation scales quadratically with the number of rooms (with a small constant), resulting in significant savings over a full geometric map. Moreover, our high-level topological representation results in 30.9% shorter trajectories in the exploration task with respect to a standard method. △ Less

Submitted 8 October, 2024; originally announced October 2024.

Comments: This manuscript has been submitted to IEEE ICRA 2025

A Control Barrier Function Candidate for Quadrotors with Limited Field of View

Authors: Biagio Trimarchi, Fabrizio Schiano, Roberto Tron

Abstract: The problem of control based on vision measurements (bearings) has been amply studied in the literature; however, the problem of addressing the limits of the field of view of physical sensors has received relatively less attention (especially for agents with non-trivial dynamics). The technical challenge is that, as in most vision-based control approaches, a standard approach to the problem requir… ▽ More The problem of control based on vision measurements (bearings) has been amply studied in the literature; however, the problem of addressing the limits of the field of view of physical sensors has received relatively less attention (especially for agents with non-trivial dynamics). The technical challenge is that, as in most vision-based control approaches, a standard approach to the problem requires knowing the distance between cameras and observed features in the scene, which is not directly available. Instead, we present a solution based on a Control Barrier Function (CBF) approach that uses a splitting of the original differential constraint to effectively remove the dependence on the unknown measurement error. Compared to the current literature, our approach gives strong robustness guarantees against bounded distance estimation errors. We showcase the proposed solution with the numerical simulations of a double integrator and a quadrotor tracking a trajectory while keeping the corners of a rectangular gate in the camera field of view. △ Less

Submitted 22 March, 2025; v1 submitted 2 October, 2024; originally announced October 2024.

Comments: 8 pages, conference paper

Navigating the Noise: A CBF Approach for Nonlinear Control with Integral Constraints

Authors: Idris Seidu, Roberto Tron

Abstract: Many physical phenomena involving mobile agents involve time-varying scalar fields, e.g., quadrotors that emit noise. As a consequence, agents can influence and can be influenced by various environmental factors such as noise. This paper delves into the challenges of controlling such agents, focusing on scenarios where we would like to prevent excessive accumulation of some quantity over select re… ▽ More Many physical phenomena involving mobile agents involve time-varying scalar fields, e.g., quadrotors that emit noise. As a consequence, agents can influence and can be influenced by various environmental factors such as noise. This paper delves into the challenges of controlling such agents, focusing on scenarios where we would like to prevent excessive accumulation of some quantity over select regions and extended trajectories. We use quadrotors that emit noise as a primary example, to regulate the trajectory of such agents in the presence of obstacles and noise emitted by the aerial vehicles themselves. First, we consider constraints that are defined over accumulated quantities, i.e functionals of the entire trajectory, as opposed to those that depend solely on the current state as in traditional Higher order Control Barrier Functions (HOCBF). Second, we propose a method to extend constraints from individual points to lines and sets by using efficient over-approximations. The efficacy of the implemented strategies is verified using simulations. Although we use quadrotors as an example, the same principles can equally apply to other scenarios, such as light emission microscopy or vehicle pollution dispersion. The technical contribution of this paper is twofold. △ Less

Submitted 18 June, 2024; originally announced June 2024.

Comments: 7 pages, 7 figures, Conference on Decision and Control(2024)

Inverse Kinematics with Vision-Based Constraints

Authors: Liangting Wu, Roberto Tron

Abstract: This paper introduces the Visual Inverse Kinematics problem (VIK) to fill the gap between robot Inverse Kinematics (IK) and visual servo control. Different from the IK problem, the VIK problem seeks to find robot configurations subject to vision-based constraints, in addition to kinematic constraints. In this work, we develop a formulation of the VIK problem with a Field of View (FoV) constraint,… ▽ More This paper introduces the Visual Inverse Kinematics problem (VIK) to fill the gap between robot Inverse Kinematics (IK) and visual servo control. Different from the IK problem, the VIK problem seeks to find robot configurations subject to vision-based constraints, in addition to kinematic constraints. In this work, we develop a formulation of the VIK problem with a Field of View (FoV) constraint, enforcing the visibility of an object from a camera on the robot. Our proposed solution is based on the idea of adding a virtual kinematic chain connecting the physical robot and the object; the FoV constraint is then equivalent to a joint angle kinematic constraint. Along the way, we introduce multiple vision-based cost functions to fulfill different objectives. We solve this formulation of the VIK problem using a method that involves a semidefinite program (SDP) constraint followed by a rank minimization algorithm. The performance of this method for solving the VIK problem is validated through simulations. △ Less

Submitted 15 June, 2024; originally announced June 2024.

Model Predictive Control for Magnetically-Actuated Cellbots

Authors: Mehdi Kermanshah, Logan E. Beaver, Max Sokolich, Fatma Ceren Kirmizitas, Sambeeta Das, Roberto Tron, Ron Weiss, Calin Belta

Abstract: This paper presents a control framework for magnetically actuated cellbots, which combines Model Predictive Control (MPC) with Gaussian Processes (GPs) as a disturbance estimator for precise trajectory tracking. To address the challenges posed by unmodeled dynamics, we integrate data-driven modeling with model-based control to accurately track desired trajectories using relatively small data. To t… ▽ More This paper presents a control framework for magnetically actuated cellbots, which combines Model Predictive Control (MPC) with Gaussian Processes (GPs) as a disturbance estimator for precise trajectory tracking. To address the challenges posed by unmodeled dynamics, we integrate data-driven modeling with model-based control to accurately track desired trajectories using relatively small data. To the best of our knowledge, this is the first work to integrate data-driven modeling with model-based control for the magnetic actuation of cellbots. The GP effectively learns and predicts unmodeled disturbances, providing uncertainty bounds as well. We validate our method through experiments with cellbots, demonstrating improved trajectory tracking accuracy. △ Less

Submitted 26 September, 2024; v1 submitted 4 June, 2024; originally announced June 2024.

TLINet: Differentiable Neural Network Temporal Logic Inference

Authors: Danyang Li, Mingyu Cai, Cristian-Ioan Vasile, Roberto Tron

Abstract: There has been a growing interest in extracting formal descriptions of the system behaviors from data. Signal Temporal Logic (STL) is an expressive formal language used to describe spatial-temporal properties with interpretability. This paper introduces TLINet, a neural-symbolic framework for learning STL formulas. The computation in TLINet is differentiable, enabling the usage of off-the-shelf gr… ▽ More There has been a growing interest in extracting formal descriptions of the system behaviors from data. Signal Temporal Logic (STL) is an expressive formal language used to describe spatial-temporal properties with interpretability. This paper introduces TLINet, a neural-symbolic framework for learning STL formulas. The computation in TLINet is differentiable, enabling the usage of off-the-shelf gradient-based tools during the learning process. In contrast to existing approaches, we introduce approximation methods for max operator designed specifically for temporal logic-based gradient techniques, ensuring the correctness of STL satisfaction evaluation. Our framework not only learns the structure but also the parameters of STL formulas, allowing flexible combinations of operators and various logical structures. We validate TLINet against state-of-the-art baselines, demonstrating that our approach outperforms these baselines in terms of interpretability, compactness, rich expressibility, and computational efficiency. △ Less

Submitted 14 May, 2024; v1 submitted 3 May, 2024; originally announced May 2024.

Learning deep Koopman operators with convex stability constraints

Authors: Marc Mitjans, Liangting Wu, Roberto Tron

Abstract: In this paper, we present a novel sufficient condition for the stability of discrete-time linear systems that can be represented as a set of piecewise linear constraints, which make them suitable for quadratic programming optimization problems. More specifically, we tackle the problem of imposing asymptotic stability to a Koopman matrix learned from data during iterative gradient descent optimizat… ▽ More In this paper, we present a novel sufficient condition for the stability of discrete-time linear systems that can be represented as a set of piecewise linear constraints, which make them suitable for quadratic programming optimization problems. More specifically, we tackle the problem of imposing asymptotic stability to a Koopman matrix learned from data during iterative gradient descent optimization processes. We show that this sufficient condition can be decoupled by rows of the system matrix, and propose a control barrier function-based projected gradient descent to enforce gradual evolution towards the stability set by running an optimization-in-the-loop during the iterative learning process. We compare the performance of our algorithm with other two recent approaches in the literature, and show that we get close to state-of-the-art performance while providing the added flexibility of allowing the optimization problem to be further customized for specific applications. △ Less

Submitted 24 April, 2024; originally announced April 2024.

Comments: 7 pages, 3 figures, 1 table, submitted to IEEE Conference on Decision and Control (CDC) 2024

Spline Trajectory Tracking and Obstacle Avoidance for Mobile Agents via Convex Optimization

Authors: Akua Dickson, Christos G. Cassandras, Roberto Tron

Abstract: We propose an output feedback control-based motion planning technique for agents to enable them to converge to a specified polynomial trajectory while imposing a set of safety constraints on our controller to avoid collisions within the free configuration space (polygonal environment). To achieve this, we 1) decompose our polygonal environment into different overlapping cells 2) write out our poly… ▽ More We propose an output feedback control-based motion planning technique for agents to enable them to converge to a specified polynomial trajectory while imposing a set of safety constraints on our controller to avoid collisions within the free configuration space (polygonal environment). To achieve this, we 1) decompose our polygonal environment into different overlapping cells 2) write out our polynomial trajectories as the output of a reference dynamical system with given initial conditions 3) formulate convergence and safety constraints as Linear Matrix Inequalities (LMIs) on our controller using Control Lyapunov Functions (CLFs) and Control Barrier Functions (CBFs) and 4) solve a semi-definite programming (SDP) problem with convergence and safety constraints imposed to synthesize a controller for each convex cell. Extensive simulations are included to test our motion planning method under different initial conditions and different reference trajectories. The synthesized controller is robust to changes in initial conditions and is always safe relative to the boundaries of the polygonal environment. △ Less

Submitted 25 March, 2024; originally announced March 2024.

Safe and Stable Teleoperation of Quadrotor UAVs under Haptic Shared Autonomy

Authors: Dawei Zhang, Roberto Tron

Abstract: We present a novel approach that aims to address both safety and stability of a haptic teleoperation system within a framework of Haptic Shared Autonomy (HSA). We use Control Barrier Functions (CBFs) to generate the control input that follows the user's input as closely as possible while guaranteeing safety. In the context of stability of the human-in-the-loop system, we limit the force feedback p… ▽ More We present a novel approach that aims to address both safety and stability of a haptic teleoperation system within a framework of Haptic Shared Autonomy (HSA). We use Control Barrier Functions (CBFs) to generate the control input that follows the user's input as closely as possible while guaranteeing safety. In the context of stability of the human-in-the-loop system, we limit the force feedback perceived by the user via a small $L_2$-gain, which is achieved by limiting the control and the force feedback via a differential constraint. Specifically, with the property of HSA, we propose two pathways to design the control and the force feedback: Sequential Control Force (SCF) and Joint Control Force (JCF). Both designs can achieve safety and stability but with different responses to the user's commands. We conducted experimental simulations to evaluate and investigate the properties of the designed methods. We also tested the proposed method on a physical quadrotor UAV and a haptic interface. △ Less

Submitted 22 March, 2024; originally announced March 2024.

Designing Robust Linear Output Feedback Controller based on CLF-CBF framework via Linear~Programming(LP-CLF-CBF)

Authors: Mahroo Bahreinian, Mehdi Kermanshah, Roberto Tron

Abstract: We consider the problem of designing output feedback controllers that use measurements from a set of landmarks to navigate through a cell-decomposable environment using duality, Control Lyapunov and Barrier Functions (CLF, CBF), and Linear Programming. We propose two objectives for navigating in an environment, one to traverse the environment by making loops and one by converging to a stabilizatio… ▽ More We consider the problem of designing output feedback controllers that use measurements from a set of landmarks to navigate through a cell-decomposable environment using duality, Control Lyapunov and Barrier Functions (CLF, CBF), and Linear Programming. We propose two objectives for navigating in an environment, one to traverse the environment by making loops and one by converging to a stabilization point while smoothing the transition between consecutive cells. We test our algorithms in a simulation environment, evaluating the robustness of the approach to practical conditions, such as bearing-only measurements, and measurements acquired with a camera with a limited field of view. △ Less

Submitted 21 March, 2024; originally announced March 2024.

Comments: arXiv admin note: text overlap with arXiv:2203.04416

Enhancing Security in Multi-Robot Systems through Co-Observation Planning, Reachability Analysis, and Network Flow

Authors: Ziqi Yang, Roberto Tron

Abstract: This paper addresses security challenges in multi-robot systems (MRS) where adversaries may compromise robot control, risking unauthorized access to forbidden areas. We propose a novel multi-robot optimal planning algorithm that integrates mutual observations and introduces reachability constraints for enhanced security. This ensures that, even with adversarial movements, compromised robots cannot… ▽ More This paper addresses security challenges in multi-robot systems (MRS) where adversaries may compromise robot control, risking unauthorized access to forbidden areas. We propose a novel multi-robot optimal planning algorithm that integrates mutual observations and introduces reachability constraints for enhanced security. This ensures that, even with adversarial movements, compromised robots cannot breach forbidden regions without missing scheduled co-observations. The reachability constraint uses ellipsoidal over-approximation for efficient intersection checking and gradient computation. To enhance system resilience and tackle feasibility challenges, we also introduce sub-teams. These cohesive units replace individual robot assignments along each route, enabling redundant robots to deviate for co-observations across different trajectories, securing multiple sub-teams without requiring modifications. We formulate the cross-trajectory co-observation plan by solving a network flow coverage problem on the checkpoint graph generated from the original unsecured MRS trajectories, providing the same security guarantees against plan-deviation attacks. We demonstrate the effectiveness and robustness of our proposed algorithm, which significantly strengthens the security of multi-robot systems in the face of adversarial threats. △ Less

Submitted 19 March, 2024; originally announced March 2024.

Comments: 12 pages, 6 figures, submitted to IEEE Transactions on Control of Network Systems

IKSPARK: An Inverse Kinematics Solver using Semidefinite Relaxation and Rank Minimization

Authors: Liangting Wu, Roberto Tron

Abstract: Inverse kinematics (IK) is a fundamental problem frequently occurred in robot control and motion planning. However, the problem is nonconvex because the kinematic map between the configuration and task spaces is generally nonlinear, which makes it challenging for fast and accurate solutions. The problem can be more complicated with the existence of different physical constraints imposed by the rob… ▽ More Inverse kinematics (IK) is a fundamental problem frequently occurred in robot control and motion planning. However, the problem is nonconvex because the kinematic map between the configuration and task spaces is generally nonlinear, which makes it challenging for fast and accurate solutions. The problem can be more complicated with the existence of different physical constraints imposed by the robot structure. In this paper, we develop an inverse kinematics solver named IKSPARK (Inverse Kinematics using Semidefinite Programming And RanK minimization) that can find solutions for robots with various structures, including open/closed kinematic chains, spherical, revolute, and/or prismatic joints. The solver works in the space of rotation matrices of the link reference frames and involves solving only convex semidefinite problems (SDPs). Specifically, the IK problem is formulated as an SDP with an additional rank-1 constraint on symmetric matrices with constant traces. The solver first solves this SDP disregarding the rank constraint to get a start point and then finds the rank-1 solution iteratively via a rank minimization algorithm with proven local convergence. Compared to other work that performs SDP relaxation for IK problems, our formulation is simpler, and uses variables with smaller sizes. We validate our approach via simulations on different robots, comparing against a standard IK method. △ Less

Submitted 18 March, 2024; originally announced March 2024.

Lyapunov Neural Network with Region of Attraction Search

Authors: Zili Wang, Sean B. Andersson, Roberto Tron

Abstract: Deep learning methods have been widely used in robotic applications, making learning-enabled control design for complex nonlinear systems a promising direction. Although deep reinforcement learning methods have demonstrated impressive empirical performance, they lack the stability guarantees that are important in safety-critical situations. One way to provide these guarantees is to learn Lyapunov… ▽ More Deep learning methods have been widely used in robotic applications, making learning-enabled control design for complex nonlinear systems a promising direction. Although deep reinforcement learning methods have demonstrated impressive empirical performance, they lack the stability guarantees that are important in safety-critical situations. One way to provide these guarantees is to learn Lyapunov certificates alongside control policies. There are three related problems: 1) verify that a given Lyapunov function candidate satisfies the conditions for a given controller on a region, 2) find a valid Lyapunov function and controller on a given region, and 3) find a valid Lyapunov function and a controller such that the region of attraction is as large as possible. Previous work has shown that if the dynamics are piecewise linear, it is possible to solve problems 1) and 2) by solving a Mixed-Integer Linear Program (MILP). In this work, we build upon this method by proposing a Lyapunov neural network that considers monotonicity over half spaces in different directions. We 1) propose a specific choice of Lyapunov function architecture that ensures non-negativity and a unique global minimum by construction, and 2) show that this can be leveraged to find the controller and Lyapunov certificates faster and with a larger valid region by maximizing the size of a square inscribed in a given level set. We apply our method to a 2D inverted pendulum, unicycle path following, a 3-D feedback system, and a 4-D cart pole system, and demonstrate it can shorten the training time by half compared to the baseline, as well as find a larger ROA. △ Less

Submitted 15 March, 2024; originally announced March 2024.

Comments: The paper has been accepted by 2024 American Control Conference (ACC)

Multi-class Temporal Logic Neural Networks

Authors: Danyang Li, Roberto Tron

Abstract: Time-series data can represent the behaviors of autonomous systems, such as drones and self-driving cars. The task of binary and multi-class classification for time-series data has become a prominent area of research. Neural networks represent a popular approach to classifying data; However, they lack interpretability, which poses a significant challenge in extracting meaningful information from t… ▽ More Time-series data can represent the behaviors of autonomous systems, such as drones and self-driving cars. The task of binary and multi-class classification for time-series data has become a prominent area of research. Neural networks represent a popular approach to classifying data; However, they lack interpretability, which poses a significant challenge in extracting meaningful information from them. Signal Temporal Logic (STL) is a formalism that describes the properties of timed behaviors. We propose a method that combines all of the above: neural networks that represent STL specifications for multi-class classification of time-series data. We offer two key contributions: 1) We introduce a notion of margin for multi-class classification, and 2) we introduce STL-based attributes for enhancing the interpretability of the results. We evaluate our method on two datasets and compare it with state-of-the-art baselines. △ Less

Submitted 24 June, 2024; v1 submitted 16 February, 2024; originally announced February 2024.

Interpretable Generative Adversarial Imitation Learning

Authors: Wenliang Liu, Danyang Li, Erfan Aasi, Roberto Tron, Calin Belta

Abstract: Imitation learning methods have demonstrated considerable success in teaching autonomous systems complex tasks through expert demonstrations. However, a limitation of these methods is their lack of interpretability, particularly in understanding the specific task the learning agent aims to accomplish. In this paper, we propose a novel imitation learning method that combines Signal Temporal Logic (… ▽ More Imitation learning methods have demonstrated considerable success in teaching autonomous systems complex tasks through expert demonstrations. However, a limitation of these methods is their lack of interpretability, particularly in understanding the specific task the learning agent aims to accomplish. In this paper, we propose a novel imitation learning method that combines Signal Temporal Logic (STL) inference and control synthesis, enabling the explicit representation of the task as an STL formula. This approach not only provides a clear understanding of the task but also allows for the incorporation of human knowledge and adaptation to new scenarios through manual adjustments of the STL formulae. Additionally, we employ a Generative Adversarial Network (GAN)-inspired training approach for both the inference and the control policy, effectively narrowing the gap between the expert and learned policies. The effectiveness of our algorithm is demonstrated through two case studies, showcasing its practical applicability and adaptability. △ Less

Submitted 15 February, 2024; originally announced February 2024.

Comments: Submitted to L4DC 2024 (under review)

Distance-coupling as an Approach to Position and Formation Control

Authors: Michael Napoli, Roberto Tron

Abstract: In this letter, we study the case of autonomous agents which are required to move to some new position based solely on the distance measured from predetermined reference points, or anchors. A novel approach, referred to as distance-coupling, is proposed for calculating the agent's position exclusively from differences between squared distance measurements. The key insight in our approach is that,… ▽ More In this letter, we study the case of autonomous agents which are required to move to some new position based solely on the distance measured from predetermined reference points, or anchors. A novel approach, referred to as distance-coupling, is proposed for calculating the agent's position exclusively from differences between squared distance measurements. The key insight in our approach is that, in doing so, we cancel out the measurement's quadratic term and obtain a function of position which is linear. We apply this method to the homing problem and prove Lyapunov stability with and without anchor placement error; identifying bounds on the region of attraction when the anchors are linearly transformed from their desired positions. As an application of the method, we show how the policy can be implemented for distributed formation control on a set of autonomous agents, proving the existence of the set of equilibria. △ Less

Submitted 13 August, 2024; v1 submitted 29 November, 2023; originally announced November 2023.

Comments: 6 pages, 7 figures

Control-Based Planning over Probability Mass Function Measurements via Robust Linear Programming

Authors: Mehdi Kermanshah, Calin Belta, Roberto Tron

Abstract: We propose an approach to synthesize linear feedback controllers for linear systems in polygonal environments. Our method focuses on designing a robust controller that can account for uncertainty in measurements. Its inputs are provided by a perception module that generates probability mass functions (PMFs) for predefined landmarks in the environment, such as distinguishable geometric features. We… ▽ More We propose an approach to synthesize linear feedback controllers for linear systems in polygonal environments. Our method focuses on designing a robust controller that can account for uncertainty in measurements. Its inputs are provided by a perception module that generates probability mass functions (PMFs) for predefined landmarks in the environment, such as distinguishable geometric features. We formulate an optimization problem with Control Lyapunov Function (CLF) and Control Barrier Function (CBF) constraints to derive a stable and safe controller. Using the strong duality of linear programs (LPs) and robust optimization, we convert the optimization problem to a linear program that can be efficiently solved offline. At a high level, our approach partially combines perception, planning, and real-time control into a single design problem. An additional advantage of our method is the ability to produce controllers capable of exhibiting nonlinear behavior while relying solely on an offline LP for control synthesis. △ Less

Submitted 12 October, 2023; originally announced October 2023.

Robustness Measures and Monitors for Time Window Temporal Logic

Authors: Ahmad Ahmad, Cristian-Ioan Vasile, Roberto Tron, Calin Belta

Abstract: Temporal logics (TLs) have been widely used to formalize interpretable tasks for cyber-physical systems. Time Window Temporal Logic (TWTL) has been recently proposed as a specification language for dynamical systems. In particular, it can easily express robotic tasks, and it allows for efficient, automata-based verification and synthesis of control policies for such systems. In this paper, we defi… ▽ More Temporal logics (TLs) have been widely used to formalize interpretable tasks for cyber-physical systems. Time Window Temporal Logic (TWTL) has been recently proposed as a specification language for dynamical systems. In particular, it can easily express robotic tasks, and it allows for efficient, automata-based verification and synthesis of control policies for such systems. In this paper, we define two quantitative semantics for this logic, and two corresponding monitoring algorithms, which allow for real-time quantification of satisfaction of formulas by trajectories of discrete-time systems. We demonstrate the new semantics and their runtime monitors on numerical examples. △ Less

Submitted 13 April, 2023; originally announced April 2023.

Comments: Submitted to the 62nd IEEE Conference on Decision and Control (CDC2023)

LQR-CBF-RRT*: Safe and Optimal Motion Planning

Authors: Guang Yang, Mingyu Cai, Ahmad Ahmad, Amanda Prorok, Roberto Tron, Calin Belta

Abstract: We present LQR-CBF-RRT*, an incremental sampling-based algorithm for offline motion planning. Our framework leverages the strength of Control Barrier Functions (CBFs) and Linear Quadratic Regulators (LQR) to generate safety-critical and optimal trajectories for a robot with dynamics described by an affine control system. CBFs are used for safety guarantees, while LQRs are employed for optimal cont… ▽ More We present LQR-CBF-RRT*, an incremental sampling-based algorithm for offline motion planning. Our framework leverages the strength of Control Barrier Functions (CBFs) and Linear Quadratic Regulators (LQR) to generate safety-critical and optimal trajectories for a robot with dynamics described by an affine control system. CBFs are used for safety guarantees, while LQRs are employed for optimal control synthesis during edge extensions. Popular CBF-based formulations for safety critical control require solving Quadratic Programs (QPs), which can be computationally expensive. Moreover, LQR-based controllers require repetitive applications of first-order Taylor approximations for nonlinear systems, which can also create an additional computational burden. To improve the motion planning efficiency, we verify the satisfaction of the CBF constraints directly in edge extension to avoid the burden of solving the QPs. We store computed optimal LQR gain matrices in a hash table to avoid re-computation during the local linearization of the rewiring procedure. Lastly, we utilize the Cross-Entropy Method for importance sampling to improve sampling efficiency. Our results show that the proposed planner surpasses its counterparts in computational efficiency and performs well in an experimental setup. △ Less

Submitted 27 September, 2023; v1 submitted 3 April, 2023; originally announced April 2023.

HoLA Robots: Mitigating Plan-Deviation Attacks in Multi-Robot Systems with Co-Observations and Horizon-Limiting Announcements

Authors: Kacper Wardega, Max von Hippel, Roberto Tron, Cristina Nita-Rotaru, Wenchao Li

Abstract: Emerging multi-robot systems rely on cooperation between humans and robots, with robots following automatically generated motion plans to service application-level tasks. Given the safety requirements associated with operating in proximity to humans and expensive infrastructure, it is important to understand and mitigate the security vulnerabilities of such systems caused by compromised robots who… ▽ More Emerging multi-robot systems rely on cooperation between humans and robots, with robots following automatically generated motion plans to service application-level tasks. Given the safety requirements associated with operating in proximity to humans and expensive infrastructure, it is important to understand and mitigate the security vulnerabilities of such systems caused by compromised robots who diverge from their assigned plans. We focus on centralized systems, where a *central entity* (CE) is responsible for determining and transmitting the motion plans to the robots, which report their location as they move following the plan. The CE checks that robots follow their assigned plans by comparing their expected location to the location they self-report. We show that this self-reporting monitoring mechanism is vulnerable to *plan-deviation attacks* where compromised robots don't follow their assigned plans while trying to conceal their movement by mis-reporting their location. We propose a two-pronged mitigation for plan-deviation attacks: (1) an attack detection technique leveraging both the robots' local sensing capabilities to report observations of other robots and *co-observation schedules* generated by the CE, and (2) a prevention technique where the CE issues *horizon-limiting announcements* to the robots, reducing their instantaneous knowledge of forward lookahead steps in the global motion plan. On a large-scale automated warehouse benchmark, we show that our solution enables attack prevention guarantees from a stealthy attacker that has compromised multiple robots. △ Less

Submitted 25 January, 2023; originally announced January 2023.

Comments: This is the long version of our paper accepted as an extended abstract to AAMAS'23

Byzantine Resilience at Swarm Scale: A Decentralized Blocklist Protocol from Inter-robot Accusations

Authors: Kacper Wardega, Max von Hippel, Roberto Tron, Cristina Nita-Rotaru, Wenchao Li

Abstract: The Weighted-Mean Subsequence Reduced (W-MSR) algorithm, the state-of-the-art method for Byzantine-resilient design of decentralized multi-robot systems, is based on discarding outliers received over Linear Consensus Protocol (LCP). Although W-MSR provides well-understood theoretical guarantees relating robust network connectivity to the convergence of the underlying consensus, the method comes wi… ▽ More The Weighted-Mean Subsequence Reduced (W-MSR) algorithm, the state-of-the-art method for Byzantine-resilient design of decentralized multi-robot systems, is based on discarding outliers received over Linear Consensus Protocol (LCP). Although W-MSR provides well-understood theoretical guarantees relating robust network connectivity to the convergence of the underlying consensus, the method comes with several limitations preventing its use at scale: (1) the number of Byzantine robots, F, to tolerate should be known a priori, (2) the requirement that each robot maintains 2F+1 neighbors is impractical for large F, (3) information propagation is hindered by the requirement that F+1 robots independently make local measurements of the consensus property in order for the swarm's decision to change, and (4) W-MSR is specific to LCP and does not generalize to applications not implemented over LCP. In this work, we propose a Decentralized Blocklist Protocol (DBP) based on inter-robot accusations. Accusations are made on the basis of locally-made observations of misbehavior, and once shared by cooperative robots across the network are used as input to a graph matching algorithm that computes a blocklist. DBP generalizes to applications not implemented via LCP, is adaptive to the number of Byzantine robots, and allows for fast information propagation through the multi-robot system while simultaneously reducing the required network connectivity relative to W-MSR. On LCP-type applications, DBP reduces the worst-case connectivity requirement of W-MSR from (2F+1)-connected to (F+1)-connected and the number of cooperative observers required to propagate new information from F+1 to just 1 observer. We demonstrate empirically that our approach to Byzantine resilience scales to hundreds of robots on cooperative target tracking, time synchronization, and localization case studies. △ Less

Submitted 17 January, 2023; originally announced January 2023.

A Graph-Based Approach to Generate Energy-Optimal Robot Trajectories in Polygonal Environments

Authors: Logan E. Beaver, Roberto Tron, Christos G. Cassandras

Abstract: As robotic systems continue to address emerging issues in areas such as logistics, mobility, manufacturing, and disaster response, it is increasingly important to rapidly generate safe and energy-efficient trajectories. In this article, we present a new approach to plan energy-optimal trajectories through cluttered environments containing polygonal obstacles. In particular, we develop a method to… ▽ More As robotic systems continue to address emerging issues in areas such as logistics, mobility, manufacturing, and disaster response, it is increasingly important to rapidly generate safe and energy-efficient trajectories. In this article, we present a new approach to plan energy-optimal trajectories through cluttered environments containing polygonal obstacles. In particular, we develop a method to quickly generate optimal trajectories for a double-integrator system, and we show that optimal path planning reduces to an integer program. To find an efficient solution, we present a distance-informed prefix search to efficiently generate optimal trajectories for a large class of environments. We demonstrate that our approach, while matching the performance of RRT* and Probabilistic Road Maps in terms of path length, outperforms both in terms of energy cost and computational time by up to an order of magnitude. We also demonstrate that our approach yields implementable trajectories in an experiment with a Crazyflie quadrotor. △ Less

Submitted 11 November, 2022; v1 submitted 9 November, 2022; originally announced November 2022.

Comments: 9 pages, 7 figures

Learning Signal Temporal Logic through Neural Network for Interpretable Classification

Authors: Danyang Li, Mingyu Cai, Cristian-Ioan Vasile, Roberto Tron

Abstract: Machine learning techniques using neural networks have achieved promising success for time-series data classification. However, the models that they produce are challenging to verify and interpret. In this paper, we propose an explainable neural-symbolic framework for the classification of time-series behaviors. In particular, we use an expressive formal language, namely Signal Temporal Logic (STL… ▽ More Machine learning techniques using neural networks have achieved promising success for time-series data classification. However, the models that they produce are challenging to verify and interpret. In this paper, we propose an explainable neural-symbolic framework for the classification of time-series behaviors. In particular, we use an expressive formal language, namely Signal Temporal Logic (STL), to constrain the search of the computation graph for a neural network. We design a novel time function and sparse softmax function to improve the soundness and precision of the neural-STL framework. As a result, we can efficiently learn a compact STL formula for the classification of time-series data through off-the-shelf gradient-based tools. We demonstrate the computational efficiency, compactness, and interpretability of the proposed method through driving scenarios and naval surveillance case studies, compared with state-of-the-art baselines. △ Less

Submitted 30 June, 2023; v1 submitted 4 October, 2022; originally announced October 2022.

Technical Report on Optimal Linear Multiple Estimation for Landmark-Based Planning via Control Synthesis

Authors: Chenfei Wang, Roberto Tron

Abstract: A common way to implement navigation in mobile robots is through the use of landmarks. In this case, the main goal of the controller is to make progress toward a goal location (stability), while avoiding the boundary of the environment (safety). In our previous work, we proposed a method to synthesize global controllers for environments with a polyhedral decomposition; our solution uses a Quadrati… ▽ More A common way to implement navigation in mobile robots is through the use of landmarks. In this case, the main goal of the controller is to make progress toward a goal location (stability), while avoiding the boundary of the environment (safety). In our previous work, we proposed a method to synthesize global controllers for environments with a polyhedral decomposition; our solution uses a Quadratically Constrained Quadratic Program with Chance Constraints to take into account the uncertainty in landmark measurements. Building upon this work, we introduce the concept of virtual landmarks, which are defined as linear combinations of actual landmark measurements that minimize the uncertainty in the resulting control actions. Interestingly, our results show that the first minimum-variance landmark is independent of the feedback control matrix, thus decoupling the design of the landmark from the one of the controller; attempting to derive additional, statistically independent landmarks, however, requires solving non-convex problems that involve also the controller. Numerical experiments demonstrate that, by minimizing the variance of the inputs, the resulting controller will be less conservative, and the robot is able to complete navigation tasks more effectively (faster and with less jitter in the trajectory). △ Less

Submitted 14 September, 2022; originally announced September 2022.

Robust Control Barrier Functions for Nonlinear Control Systems with Uncertainty: A Duality-based Approach

Authors: Max H. Cohen, Calin Belta, Roberto Tron

Abstract: This paper studies the design of controllers that guarantee stability and safety of nonlinear control affine systems with parametric uncertainty in both the drift and control vector fields. To this end, we introduce novel classes of robust control barrier functions (RCBF) and robust control Lyapunov functions (RCLF) that facilitate the synthesis of safety-critical controllers in the presence of pa… ▽ More This paper studies the design of controllers that guarantee stability and safety of nonlinear control affine systems with parametric uncertainty in both the drift and control vector fields. To this end, we introduce novel classes of robust control barrier functions (RCBF) and robust control Lyapunov functions (RCLF) that facilitate the synthesis of safety-critical controllers in the presence of parametric uncertainty using quadratic programming. Since the initial bounds on the system uncertainty may be highly conservative, we present a data-driven approach to reducing such bounds using input-output data collected online. In particular, we leverage an integral set-membership identification algorithm that iteratively shrinks the set of possible system parameters online and guarantees stability and safety during learning. The efficacy of the developed approach is illustrated on two numerical examples. △ Less

Submitted 11 August, 2022; originally announced August 2022.

Comments: To appear at the 2022 IEEE Conference on Decision and Control

Adaptive Sampling-based Motion Planning with Control Barrier Functions

Authors: Ahmad Ahmad, Calin Belta, Roberto Tron

Abstract: Sampling-based algorithms, such as Rapidly Exploring Random Trees (RRT) and its variants, have been used extensively for motion planning. Control barrier functions (CBFs) have been recently proposed to synthesize controllers for safety-critical systems. In this paper, we combine the effectiveness of RRT-based algorithms with the safety guarantees provided by CBFs in a method called CBF-RRT$^\ast$.… ▽ More Sampling-based algorithms, such as Rapidly Exploring Random Trees (RRT) and its variants, have been used extensively for motion planning. Control barrier functions (CBFs) have been recently proposed to synthesize controllers for safety-critical systems. In this paper, we combine the effectiveness of RRT-based algorithms with the safety guarantees provided by CBFs in a method called CBF-RRT$^\ast$. CBFs are used for local trajectory planning for RRT$^\ast$, avoiding explicit collision checking of the extended paths. We prove that CBF-RRT$^\ast$ preserves the probabilistic completeness of RRT$^\ast$. Furthermore, in order to improve the sampling efficiency of the algorithm, we equip the algorithm with an adaptive sampling procedure, which is based on the cross-entropy method (CEM) for importance sampling (IS). The procedure exploits the tree of samples to focus the sampling in promising regions of the configuration space. We demonstrate the efficacy of the proposed algorithms through simulation examples. △ Less

Submitted 1 June, 2022; originally announced June 2022.

Comments: Submitted to CDC2022

Output-Feedback Path Planning with Robustness to State-Dependent Errors

Authors: Mahroo Bahreinian, Roberto Tron

Abstract: We consider the problem of sample-based feedback motion planning from measurements affected by systematic errors. Our previous work presented output feedback controllers that use measurements from landmarks in the environment to navigate through a cell-decomposable environment using duality, Control Lyapunov and Barrier Functions (CLF, CBF), and Linear Programming. In this paper, we build on this… ▽ More We consider the problem of sample-based feedback motion planning from measurements affected by systematic errors. Our previous work presented output feedback controllers that use measurements from landmarks in the environment to navigate through a cell-decomposable environment using duality, Control Lyapunov and Barrier Functions (CLF, CBF), and Linear Programming. In this paper, we build on this previous work with a novel strategy that allows the use of measurements affected by systematic errors in perceived depth (similarly to what might be generated by vision-based sensors), as opposed to accurate displacement measurements. As a result, our new method has the advantage of providing more robust performance (with quantitative guarantees) when inaccurate sensors are used. We test the proposed algorithm in the simulation to evaluate the performance limits of our approach predicted by our theoretical derivations. △ Less

Submitted 15 May, 2022; originally announced May 2022.

Comments: This work has been submitted to the CDC conference 2022. arXiv admin note: text overlap with arXiv:2203.04416

Koopman pose predictions for temporally consistent human walking estimations

Authors: Marc Mitjans, David M. Levine, Louis N. Awad, Roberto Tron

Abstract: We tackle the problem of tracking the human lower body as an initial step toward an automatic motion assessment system for clinical mobility evaluation, using a multimodal system that combines Inertial Measurement Unit (IMU) data, RGB images, and point cloud depth measurements. This system applies the factor graph representation to an optimization problem that provides 3-D skeleton joint estimatio… ▽ More We tackle the problem of tracking the human lower body as an initial step toward an automatic motion assessment system for clinical mobility evaluation, using a multimodal system that combines Inertial Measurement Unit (IMU) data, RGB images, and point cloud depth measurements. This system applies the factor graph representation to an optimization problem that provides 3-D skeleton joint estimations. In this paper, we focus on improving the temporal consistency of the estimated human trajectories to greatly extend the range of operability of the depth sensor. More specifically, we introduce a new factor graph factor based on Koopman theory that embeds the nonlinear dynamics of several lower-limb movement activities. This factor performs a two-step process: first, a custom activity recognition module based on spatial temporal graph convolutional networks recognizes the walking activity; then, a Koopman pose prediction of the subsequent skeleton is used as an a priori estimation to drive the optimization problem toward more consistent results. We tested the performance of this module on datasets composed of multiple clinical lowerlimb mobility tests, and we show that our approach reduces outliers on the skeleton form by almost 1 m, while preserving natural walking trajectories at depths up to more than 10 m. △ Less

Submitted 5 November, 2022; v1 submitted 5 May, 2022; originally announced May 2022.

Comments: 8 pages, 6 figures, accepted to IROS 2022

Bearing-Based Formation Control with Optimal Motion Trajectory

Authors: Zili Wang, Sean B. Andersson, Roberto Tron

Abstract: Bearing-based distributed formation control is attractive because it can be implemented using vision-based measurements to achieve a desired formation. Gradient-descent-based controllers using bearing measurements have been shown to have many beneficial characteristics, such as global convergence, applicability to different graph topologies and workspaces of arbitrary dimension, and some flexibili… ▽ More Bearing-based distributed formation control is attractive because it can be implemented using vision-based measurements to achieve a desired formation. Gradient-descent-based controllers using bearing measurements have been shown to have many beneficial characteristics, such as global convergence, applicability to different graph topologies and workspaces of arbitrary dimension, and some flexibility in the choice of the cost. In practice, however, such controllers typically yield convoluted paths from their initial location to the final position in the formation. In this paper we propose a novel procedure to optimize gradient-descent-based bearing-based formation controllers to obtain shorter paths. Our approach is based on the parameterization of the cost function and, by extension, of the controller. We form and solve a nonlinear optimization problem with the sum of path lengths of the agent trajectories as the objective and subject to the original equilibria and global convergence conditions for formation control. Our simulation shows that the parameters can be optimized from a very small number of training samples (1 to 7) to straighten the trajectory by around 16% for a large number of random initial conditions for bearing-only formation. However, in the absence of any range information, the scale of the formation is not fixed and this optimization may lead to an undesired compression of the formation size. Including range measurements avoids this issue and leads to further trajectories straightening by 66%. △ Less

Submitted 23 March, 2022; v1 submitted 22 March, 2022; originally announced March 2022.

Comments: The paper has been accepted by 2022 American Control Conference (ACC)

Sample-Based Output-Feedback Navigation with Bearing Measurements

Authors: Mahroo Bahreinian, Marc Mitjans, Roy Xing, Roberto Tron

Abstract: We consider the problem of sample-based feedback-based motion planning from bearing (direction-only) measurements. We build on our previous work that defines a cell decomposition of the environment using RRT*, and finds an output feedback controller to navigate through each cell toward a goal location using duality, Control Lyapunov and Barrier Functions (CLF, CBF), and Linear Programming. In this… ▽ More We consider the problem of sample-based feedback-based motion planning from bearing (direction-only) measurements. We build on our previous work that defines a cell decomposition of the environment using RRT*, and finds an output feedback controller to navigate through each cell toward a goal location using duality, Control Lyapunov and Barrier Functions (CLF, CBF), and Linear Programming. In this paper, we propose a novel strategy that uses relative bearing measurements with respect to a set of landmarks in the environment, as opposed to full relative displacements. The main advantage is then that the measurements can be obtained using a simple monocular camera. We test the proposed algorithm in the simulation, and then in an experimental environment to evaluate the performance of our approach with respect to practical issues such as mismatches in the dynamical model of the robot, and measurements acquired with a camera with a limited field of view. △ Less

Submitted 8 March, 2022; originally announced March 2022.

Comments: This work is submitted to the IROS 2022 conference

Robust Sample-Based Output-Feedback Path Planning

Authors: Mahroo Bahreinian, Marc Mitjans, Roberto Tron

Abstract: We propose a novel approach for sampling-based and control-based motion planning that combines a representation of the environment obtained via a modified version of optimal Rapidly-exploring Random Trees (RRT*), with landmark-based output-feedback controllers obtained via Control Lyapunov Functions, Control Barrier Functions, and robust Linear Programming. Our solution inherits many benefits of R… ▽ More We propose a novel approach for sampling-based and control-based motion planning that combines a representation of the environment obtained via a modified version of optimal Rapidly-exploring Random Trees (RRT*), with landmark-based output-feedback controllers obtained via Control Lyapunov Functions, Control Barrier Functions, and robust Linear Programming. Our solution inherits many benefits of RRT*-like algorithms, such as the ability to implicitly handle arbitrarily complex obstacles, and asymptotic optimality. Additionally, it extends planning beyond the discrete nominal paths, as feedback controllers can correct deviations from such paths, and are robust to discrepancies between the map used for planning and the real environment. We test our algorithms first in simulations and then in experiments, testing the robustness of the approach to practical conditions, such as deformations of the environment, mismatches in the dynamical model of the robot, and measurements acquired with a camera with a limited field of view. △ Less

Submitted 28 May, 2021; originally announced May 2021.

Comments: submitted to IROS2021

Stable Haptic Teleoperation of UAVs via Small $L_2$ Gain and Control Barrier Functions

Authors: Dawei Zhang, Roberto Tron

Abstract: We present a novel haptic teleoperation approach that considers not only the safety but also the stability of a teleoperation system. Specifically, we build upon previous work on haptic shared control, which uses control barrier functions (CBFs) to generate a reference haptic feedback that informs the human operator on the internal state of the system, helping them to safely navigate the robot wit… ▽ More We present a novel haptic teleoperation approach that considers not only the safety but also the stability of a teleoperation system. Specifically, we build upon previous work on haptic shared control, which uses control barrier functions (CBFs) to generate a reference haptic feedback that informs the human operator on the internal state of the system, helping them to safely navigate the robot without taking away their control authority. Crucially, in this approach the force rendered to the user is not directly reflected in the motion of the robot (which is still directly controlled by the user); however, previous work in the area neglected to consider the feedback loop through the user, possibly resulting in unstable closed trajectories. In this paper we introduce a differential constraint on the rendered force that makes the system finite-gain $L_2$ stable; the constraint results in a Quadratically Constrained Quadratic Program (QCQP), for which we provide a closed-form solution. Our constraint is related to but less restrictive than the typical passivity constraint used in previous literature. We conducted an experimental simulation in which a human operator flies a UAV near an obstacle to evaluate the proposed method. △ Less

Submitted 22 March, 2021; originally announced March 2021.

Haptic Feedback Improves Human-Robot Agreement and User Satisfaction in Shared-Autonomy Teleoperation

Authors: Dawei Zhang, Roberto Tron, Rebecca P. Khurshid

Abstract: Shared autonomy teleoperation can guarantee safety, but does so by reducing the human operator's control authority, which can lead to reduced levels of human-robot agreement and user satisfaction. This paper presents a novel haptic shared autonomy teleoperation paradigm that uses haptic feedback to inform the user about the inner state of a shared autonomy paradigm, while still guaranteeing safety… ▽ More Shared autonomy teleoperation can guarantee safety, but does so by reducing the human operator's control authority, which can lead to reduced levels of human-robot agreement and user satisfaction. This paper presents a novel haptic shared autonomy teleoperation paradigm that uses haptic feedback to inform the user about the inner state of a shared autonomy paradigm, while still guaranteeing safety. This differs from haptic shared control, which uses haptic feedback to inform the user's actions, but gives the human operator full control over the robot's actions. We conducted a user study in which twelve users flew a simulated UAV in a search-and-rescue task with no assistance or assistance provided by haptic shared control, shared autonomy, or haptic shared autonomy. All assistive teleoperation methods use control barrier functions to find a control command that is both safe and as close as possible to the human-generated control command. For assistive teleoperation conditions with haptic feedback, we apply a force to the user that is proportional to the difference between the human-generated control and the safe control. We find that haptic shared autonomy improves the user's task performance and satisfaction. We also find that haptic feedback in assistive teleoperation can improve the user's situational awareness. Finally, results show that adding haptic feedback to shared-autonomy teleoperation can improve human-robot agreement. △ Less

Submitted 4 March, 2021; originally announced March 2021.

Chance Constraint Robust Control with Control Barrier Functions

Authors: Chenfei Wang, Mahroo Bahreinian, Roberto Tron

Abstract: In this paper, we propose a novel approach to synthesize linear feedback controllers for navigating in polygonal environments using noisy measurements and a convex cell decomposition. Our method is based on formulating chance constraints for the convergence and collision avoidance condition. In particular, the stability and safety guarantees come from chance Control Barrier Function (CBF) constrai… ▽ More In this paper, we propose a novel approach to synthesize linear feedback controllers for navigating in polygonal environments using noisy measurements and a convex cell decomposition. Our method is based on formulating chance constraints for the convergence and collision avoidance condition. In particular, the stability and safety guarantees come from chance Control Barrier Function (CBF) constraints and chance Control Lyapunov Function (CLF) constraints, respectively. We use convex over-approximations to get upper bounds of the constraints, leading to a convex robust quadratic program for finding the controller. We apply and provide simulation results for equilibrium control and path control. The result shows that the controller is robust with the noise input. △ Less

Submitted 18 December, 2020; originally announced December 2020.

Bearing-Only Navigation with Field of View Constraints

Authors: Arman Karimian, Roberto Tron

Abstract: This paper addresses the problem of navigation using only relative direction measurements (i.e., relative distances are unknown) under field of view constraints. We present a novel navigation vector field for the bearing-based visual homing problem with respect to static visual landmarks in 2-D and 3-D environments. Our method employs two control fields that are tangent and normal to ellipsoids ha… ▽ More This paper addresses the problem of navigation using only relative direction measurements (i.e., relative distances are unknown) under field of view constraints. We present a novel navigation vector field for the bearing-based visual homing problem with respect to static visual landmarks in 2-D and 3-D environments. Our method employs two control fields that are tangent and normal to ellipsoids having landmarks as their foci. The tangent field steers the robot to a set of points where the average of observed bearings is parallel to the average of the desired bearings, and the normal field uses the angle between a pair of bearings as a proxy to adjust the robot's distance from landmarks and to satisfy the field of view constraints. Both fields are blended together to construct an almost globally stable control law. Our method is easy to implement, as it requires only comparisons between average bearings, and between angles of pairs of vectors. We provide simulations that demonstrate the performance of our approach for a double integrator system and unicycles. △ Less

Submitted 29 December, 2020; v1 submitted 15 September, 2020; originally announced September 2020.

Report number: main.cssl.20-0736.9c625022

Non-uniform Sampled Motion Planning for Continuous-time STL

Authors: Guang Yang, Calin Belta, Roberto Tron

Abstract: This paper presents an offline motion planner for linear cyber-physical systems that satisfy a continuous-time Signal Temporal Logic (STL) specification, in which controls are applied in a Zeroth-order Hold (ZOH) manner. The motion planning problem is formulated as a Mixed-integer Program (MIP) with nonuniform control updates. We develop a novel method to obtain bounds of Control Barrier Functions… ▽ More This paper presents an offline motion planner for linear cyber-physical systems that satisfy a continuous-time Signal Temporal Logic (STL) specification, in which controls are applied in a Zeroth-order Hold (ZOH) manner. The motion planning problem is formulated as a Mixed-integer Program (MIP) with nonuniform control updates. We develop a novel method to obtain bounds of Control Barrier Functions (CBF) and linear predicates to render both spatial and temporal requirements. The theoretical results are validated in numerical examples. △ Less

Submitted 3 June, 2020; originally announced June 2020.

Statistical Outlier Identification in Multi-robot Visual SLAM using Expectation Maximization

Authors: Arman Karimian, Ziqi Yang, Roberto Tron

Abstract: This paper introduces a novel and distributed method for detecting inter-map loop closure outliers in simultaneous localization and mapping (SLAM). The proposed algorithm does not rely on a good initialization and can handle more than two maps at a time. In multi-robot SLAM applications, maps made by different agents have nonidentical spatial frames of reference which makes initialization very dif… ▽ More This paper introduces a novel and distributed method for detecting inter-map loop closure outliers in simultaneous localization and mapping (SLAM). The proposed algorithm does not rely on a good initialization and can handle more than two maps at a time. In multi-robot SLAM applications, maps made by different agents have nonidentical spatial frames of reference which makes initialization very difficult in the presence of outliers. This paper presents a probabilistic approach for detecting incorrect orientation measurements prior to pose graph optimization by checking the geometric consistency of rotation measurements. Expectation-Maximization is used to fine-tune the model parameters. As ancillary contributions, a new approximate discrete inference procedure is presented which uses evidence on loops in a graph and is based on optimization (Alternate Direction Method of Multipliers). This method yields superior results compared to Belief Propagation and has convergence guarantees. Simulation and experimental results are presented that evaluate the performance of the outlier detection method and the inference algorithm on synthetic and real-world data. △ Less

Submitted 7 February, 2020; originally announced February 2020.

Distributed and Consistent Multi-Image Feature Matching via QuickMatch

Authors: Zachary Serlin, Guang Yang, Brandon Sookraj, Calin Belta, Roberto Tron

Abstract: In this work we consider the multi-image object matching problem, extend a centralized solution of the problem to a distributed solution, and present an experimental application of the centralized solution. Multi-image feature matching is a keystone of many applications, including simultaneous localization and mapping, homography, object detection, and structure from motion. We first review the Qu… ▽ More In this work we consider the multi-image object matching problem, extend a centralized solution of the problem to a distributed solution, and present an experimental application of the centralized solution. Multi-image feature matching is a keystone of many applications, including simultaneous localization and mapping, homography, object detection, and structure from motion. We first review the QuickMatch algorithm for multi-image feature matching. We then present a scheme for distributing sets of features across computational units (agents) that largely preserves feature match quality and minimizes communication between agents (avoiding, in particular, the need of flooding all data to all agents). Finally, we show how QuickMatch performs on an object matching test with low quality images. The centralized QuickMatch algorithm is compared to other standard matching algorithms, while the Distributed QuickMatch algorithm is compared to the centralized algorithm in terms of preservation of match consistency. The presented experiment shows that QuickMatch matches features across a large number of images and features in larger numbers and more accurately than standard techniques. △ Less

Submitted 29 October, 2019; originally announced October 2019.

Robust Planning and Control For Polygonal Environments via Linear Programming

Authors: Mahroo Bahreinian, Erfan Aasi, Roberto Tron

Abstract: We propose a novel approach for navigating in polygonal environments by synthesizing controllers that take as input relative displacement measurements with respect to a set of landmarks. Our algorithm is based on solving a sequence of robust min-max Linear Programming problems on the elements of a cell decomposition of the environment. The optimization problems are formulated using linear Control… ▽ More We propose a novel approach for navigating in polygonal environments by synthesizing controllers that take as input relative displacement measurements with respect to a set of landmarks. Our algorithm is based on solving a sequence of robust min-max Linear Programming problems on the elements of a cell decomposition of the environment. The optimization problems are formulated using linear Control Lyapunov Function (CLF) and Control Barrier Function (CBF) constraints, to provide stability and safety guarantees, respectively. The inner maximization problem ensures that these constraints are met by all the points in each cell, while the outer minimization problem balances the different constraints in a robust way. We show that the min-max optimization problems can be solved efficiently by transforming it into regular linear programming via the dualization of the inner maximization problem. We test our algorithm to agents with first and second-order integrator dynamics, although our approach is in principle applicable to any system with piecewise linear dynamics. Through our theoretical results and simulations, we show that the resulting controllers: are optimal (with respect to the criterion used in the formulation), are applicable to linear systems of any order, are robust to changes to the start location (since they do not rely on a single nominal path), and to significant deformations of the environment. △ Less

Submitted 11 October, 2020; v1 submitted 17 October, 2019; originally announced October 2019.

Comments: Second version of the paper. This version was submitted to ACC 2021

A Computational Theory of Robust Localization Verifiability in the Presence of Pure Outlier Measurements

Authors: Mahroo Bahreinian, Roberto Tron

Abstract: The problem of localizing a set of nodes from relative pairwise measurements is at the core of many applications such as Structure from Motion (SfM), sensor networks, and Simultaneous Localization And Mapping (SLAM). In practical situations, the accuracy of the relative measurements is marred by noise and outliers; hence, we have the problem of quantifying how much we should trust the solution ret… ▽ More The problem of localizing a set of nodes from relative pairwise measurements is at the core of many applications such as Structure from Motion (SfM), sensor networks, and Simultaneous Localization And Mapping (SLAM). In practical situations, the accuracy of the relative measurements is marred by noise and outliers; hence, we have the problem of quantifying how much we should trust the solution returned by some given localization solver. In this work, we focus on the question of whether an L1-norm robust optimization formulation can recover a solution that is identical to the ground truth, under the scenario of translation-only measurements corrupted exclusively by outliers and no noise; we call this concept verifiability. On the theoretical side, we prove that the verifiability of a problem depends only on the topology of the graph of measurements, the edge support of the outliers, and their signs, while it is independent of ground truth locations of the nodes, and of any positive scaling of the outliers. On the computational side, we present a novel approach based on the dual simplex algorithm that can check the verifiability of a problem, completely characterize the space of equivalent solutions if they exist, and identify subgraphs that are verifiable. As an application of our theory, we provide a procedure to compute a priori probability of recovering a solution congruent or equivalent to the ground truth given a measurement graph and the probabilities of each edge containing an outlier. △ Less

Submitted 12 October, 2019; originally announced October 2019.

Comments: Accepted for publication in the proceedings of Conference on Decision and Control 2019

Bearing-Only Consensus and Formation Control under Directed Topologies

Authors: Arman Karimian, Roberto Tron

Abstract: We address the problems of bearing-only consensus and formation control, where each agent can only measure the relative bearings of its neighbors and relative distances are not available. We provide stability results for the Filippov solutions of two gradient-descent laws from non-smooth Lyapunov functions in the context of differential inclusion. For the consensus and formation control problems w… ▽ More We address the problems of bearing-only consensus and formation control, where each agent can only measure the relative bearings of its neighbors and relative distances are not available. We provide stability results for the Filippov solutions of two gradient-descent laws from non-smooth Lyapunov functions in the context of differential inclusion. For the consensus and formation control problems with undirected sensing topologies, we prove finite-time and asymptotic convergence of the proposed non-smooth gradient flows. For the directed consensus problem, we prove asymptotic convergence using a different non-smooth Lyapunov function given that the sensing graph has a globally reachable node. Finally, For the directed formation control problem we prove asymptotic convergence for directed cycles and directed acyclic graphs and also introduce a new notion of bearing persistence which guarantees convergence to the desired bearings. △ Less

Submitted 30 September, 2019; originally announced October 2019.

Sampling-based Motion Planning via Control Barrier Functions

Authors: Guang Yang, Bee Vang, Zachary Serlin, Calin Belta, Roberto Tron

Abstract: Robot motion planning is central to real-world autonomous applications, such as self-driving cars, persistence surveillance, and robotic arm manipulation. One challenge in motion planning is generating control signals for nonlinear systems that result in obstacle free paths through dynamic environments. In this paper, we propose Control Barrier Function guided Rapidly-exploring Random Trees (CBF-R… ▽ More Robot motion planning is central to real-world autonomous applications, such as self-driving cars, persistence surveillance, and robotic arm manipulation. One challenge in motion planning is generating control signals for nonlinear systems that result in obstacle free paths through dynamic environments. In this paper, we propose Control Barrier Function guided Rapidly-exploring Random Trees (CBF-RRT), a sampling-based motion planning algorithm for continuous-time nonlinear systems in dynamic environments. The algorithm focuses on two objectives: efficiently generating feasible controls that steer the system toward a goal region, and handling environments with dynamical obstacles in continuous time. We formulate the control synthesis problem as a Quadratic Program (QP) that enforces Control Barrier Function (CBF) constraints to achieve obstacle avoidance. Additionally, CBF-RRT does not require nearest neighbor or collision checks when sampling, which greatly reduce the run-time overhead when compared to standard RRT variants. △ Less

Submitted 15 July, 2019; originally announced July 2019.

Continuous-time Signal Temporal Logic Planning with Control Barrier Function

Authors: Guang Yang, Roberto Tron, Calin Belta

Abstract: Temporal Logic (TL) guided control problems have gained interests in recent years. By using the TL, one can specify a wide range of temporal constraints on the system and is widely used in cyber-physical systems. On the other hand, Control Barrier Functions have also gained interests in the context of safety critical applications. However, most of the existing approaches only focus on discrete-tim… ▽ More Temporal Logic (TL) guided control problems have gained interests in recent years. By using the TL, one can specify a wide range of temporal constraints on the system and is widely used in cyber-physical systems. On the other hand, Control Barrier Functions have also gained interests in the context of safety critical applications. However, most of the existing approaches only focus on discrete-time dynamical systems. In this paper, we propose an offline trajectory planner for linear systems subject to safety and temporal specifications. Such specifications can be expressed as logical junctions or disjunctions of linear CBFs, or as STL specifications with linear predicates. Our planner produces trajectories that are valid in continuous time, while assuming only discrete-time control updates and arbitrary time interval in the STL formula. Our planner is based on a Mixed Integer Quadratic Programming (MIQP) formulation, where the linear STL predicates are encoded as set of linear constraints to guarantee satisfaction at on a finite discrete set of time instants, while we use CBFs to derive constraints that guarantee continuous satisfaction between time instants. Moreover, we have shown the predicates can be encoded as time-based CBF constraints for system with any relative degrees. We validate our theoretical results and formulation through numerical simulations. △ Less

Submitted 9 March, 2019; originally announced March 2019.

Self-triggered Control for Safety Critical Systems using Control Barrier Functions

Authors: Guang Yang, Calin Belta, Roberto Tron

Abstract: We propose a real-time control strategy that combines self-triggered control with Control Lyapunov Functions (CLF) and Control Barrier Functions (CBF). Similar to related works proposing CLF-CBF-based controllers, the computation of the controller is achieved by solving a Quadratic Program (QP). However, we propose a Zeroth-Order Hold (ZOH) implementation of the controller that overcomes the main… ▽ More We propose a real-time control strategy that combines self-triggered control with Control Lyapunov Functions (CLF) and Control Barrier Functions (CBF). Similar to related works proposing CLF-CBF-based controllers, the computation of the controller is achieved by solving a Quadratic Program (QP). However, we propose a Zeroth-Order Hold (ZOH) implementation of the controller that overcomes the main limitations of traditional approaches based on periodic controllers, i.e., unnecessary controller updates and potential violations of the safety constraints. Central to our approach is the novel notion of safe period, which enforces a strong safety guarantee for implementing ZOH control. In addition, we prove that the system does not exhibit a Zeno behavior as it approaches the desired equilibrium. △ Less

Submitted 8 March, 2019; originally announced March 2019.

Non-Natural Metrics on the Tangent Bundle

Authors: Bee Vang, Roberto Tron

Abstract: Natural metrics provide a way to induce a metric on the tangent bundle from the metric on its base manifold. The most studied type is the Sasaki metric, which applies the base metric separately to the vertical and horizontal components. We study a more general class of metrics which introduces interactions between the vertical and horizontal components, with scalar weights. Additionally, we explic… ▽ More Natural metrics provide a way to induce a metric on the tangent bundle from the metric on its base manifold. The most studied type is the Sasaki metric, which applies the base metric separately to the vertical and horizontal components. We study a more general class of metrics which introduces interactions between the vertical and horizontal components, with scalar weights. Additionally, we explicitly clarify how to apply our and other induced metrics on the tangent bundle to vector fields where the vertical component is not constant along the fibers. We give application to the Special Orthogonal Group SO(3) as an example. △ Less

Submitted 18 September, 2018; originally announced September 2018.

Comments: 9 Pages

A Factorization Approach to Inertial Affine Structure from Motion

Authors: Roberto Tron

Abstract: We consider the problem of reconstructing a 3-D scene from a moving camera with high frame rate using the affine projection model. This problem is traditionally known as Affine Structure from Motion (Affine SfM), and can be solved using an elegant low-rank factorization formulation. In this paper, we assume that an accelerometer and gyro are rigidly mounted with the camera, so that synchronized li… ▽ More We consider the problem of reconstructing a 3-D scene from a moving camera with high frame rate using the affine projection model. This problem is traditionally known as Affine Structure from Motion (Affine SfM), and can be solved using an elegant low-rank factorization formulation. In this paper, we assume that an accelerometer and gyro are rigidly mounted with the camera, so that synchronized linear acceleration and angular velocity measurements are available together with the image measurements. We extend the standard Affine SfM algorithm to integrate these measurements through the use of image derivatives. △ Less

Submitted 8 August, 2016; originally announced August 2016.