Showing 1–14 of 14 results for author: Julius, A

Iterative Motion Planning in Multi-agent Systems with Opportunistic Communication under Disturbance

Authors: Neelanga Thelasingha, Agung Julius, James Humann, James Dotterweich

Abstract: In complex multi-agent systems involving heterogeneous teams, uncertainty arises from numerous sources like environmental disturbances, model inaccuracies, and changing tasks. This causes planned trajectories to become infeasible, requiring replanning. Further, different communication architectures used in multi-agent systems give rise to asymmetric knowledge of planned trajectories across the age… ▽ More In complex multi-agent systems involving heterogeneous teams, uncertainty arises from numerous sources like environmental disturbances, model inaccuracies, and changing tasks. This causes planned trajectories to become infeasible, requiring replanning. Further, different communication architectures used in multi-agent systems give rise to asymmetric knowledge of planned trajectories across the agents. In such systems, replanning must be done in a communication-aware fashion. This paper establishes the conditions for synchronization and feasibility in epistemic planning scenarios introduced by opportunistic communication architectures. We also establish conditions on task satisfaction based on quantified recoverability of disturbances in an iterative planning scheme. We further validate these theoretical results experimentally in a UAV--UGV task assignment problem. △ Less

Submitted 16 March, 2025; originally announced March 2025.

Analysis of human steering behavior differences in human-in-control and autonomy-in-control driving

Authors: Rene Mai, Agung Julius, Sandipan Mishra

Abstract: Steering models (such as the generalized two-point model) predict human steering behavior well when the human is in direct control of a vehicle. In vehicles under autonomous control, human control inputs are not used; rather, an autonomous controller applies steering and acceleration commands to the vehicle. For example, human steering input may be used for state estimation rather than direct cont… ▽ More Steering models (such as the generalized two-point model) predict human steering behavior well when the human is in direct control of a vehicle. In vehicles under autonomous control, human control inputs are not used; rather, an autonomous controller applies steering and acceleration commands to the vehicle. For example, human steering input may be used for state estimation rather than direct control. We show that human steering behavior changes when the human no longer directly controls the vehicle and the two are instead working in a shared autonomy paradigm. Thus, when a vehicle is not under direct human control, steering models like the generalized two-point model do not predict human steering behavior. We also show that the error between predicted human steering behavior and actual human steering behavior reflects a fundamental difference when the human directly controls the vehicle compared to when the vehicle is autonomously controlled. Moreover, we show that a single distribution describes the error between predicted human steering behavior and actual human steering behavior when the human's steering inputs are used for state estimation and the vehicle is autonomously controlled, indicating there may be a underlying model for human steering behavior under this type of shared autonomous control. Future work includes determining this shared autonomous human steering model and demonstrating its performance. △ Less

Submitted 30 September, 2024; originally announced October 2024.

Comments: 6 pages, 10 figures, accepted for publication at the 5th IFAC at the 5th IFAC Workshop on Cyber-Physical Human Systems

Open-Source Software Architecture for Multi-Robot Wire Arc Additive Manufacturing (WAAM)

Authors: Honglu He, Chen-lung Lu, Jinhan Ren, Joni Dhar, Glenn Saunders, John Wason, Johnson Samuel, Agung Julius, John T. Wen

Abstract: Wire Arc Additive Manufacturing (WAAM) is a metal 3D printing technology that deposits molten metal wire on a substrate to form desired geometries. Articulated robot arms are commonly used in WAAM to produce complex geometric shapes. However, they mostly rely on proprietary robot and weld control software that limits process tuning and customization, incorporation of third-party sensors, implement… ▽ More Wire Arc Additive Manufacturing (WAAM) is a metal 3D printing technology that deposits molten metal wire on a substrate to form desired geometries. Articulated robot arms are commonly used in WAAM to produce complex geometric shapes. However, they mostly rely on proprietary robot and weld control software that limits process tuning and customization, incorporation of third-party sensors, implementation on robots and weld controllers from multiple vendors, and customizable user programming. This paper presents a general open-source software architecture for WAAM that addresses these limitations. The foundation of this architecture is Robot Raconteur, an open-source control and communication framework that serves as the middleware for integrating robots and sensors from different vendors. Based on this architecture, we developed an end-to-end robotic WAAM implementation that takes a CAD file to a printed WAAM part and evaluates the accuracy of the result. The major components in the architecture include part slicing, robot motion planning, part metrology, in-process sensing, and process tuning. The current implementation is based on Motoman robots and Fronius weld controller, but the approach is applicable to other industrial robots and weld controllers. The capability of the WAAM tested is demonstrated through the printing of parts of various geometries and acquisition of in-process sensor data for motion adjustment. △ Less

Submitted 7 August, 2024; originally announced August 2024.

Generalized two-point visual control model of human steering for accurate state estimation

Authors: Rene Mai, Katherine Sears, Grace Roessling, Agung Julius, Sandipan Mishra

Abstract: We derive and validate a generalization of the two-point visual control model, an accepted cognitive science model for human steering behavior. The generalized model is needed as current steering models are either insufficiently accurate or too complex for online state estimation. We demonstrate that the generalized model replicates specific human steering behavior with high precision (85\% reduct… ▽ More We derive and validate a generalization of the two-point visual control model, an accepted cognitive science model for human steering behavior. The generalized model is needed as current steering models are either insufficiently accurate or too complex for online state estimation. We demonstrate that the generalized model replicates specific human steering behavior with high precision (85\% reduction in modeling error) and integrate this model into a human-as-advisor framework where human steering inputs are used for state estimation. As a benchmark study, we use this framework to decipher ambiguous lane markings represented by biased lateral position measurements. We demonstrate that, with the generalized model, the state estimator can accurately estimate the true vehicle state, providing lateral state estimates with under 0.25 m error on average across participants. However, without the generalized model, the estimator cannot accurately estimate the vehicle's lateral state. △ Less

Submitted 5 June, 2024; originally announced June 2024.

Comments: 6 pages, 9 figures, This work has been submitted to IFAC for possible publication

Fast and Accurate Relative Motion Tracking for Dual Industrial Robots

Authors: Honglu He, Chen-lung Lu, Glenn Saunders, Pinghai Yang, Jeffrey Schoonover, Leo Ajdelsztajn, John Wason, Santiago Paternain, Agung Julius, John T. Wen

Abstract: Industrial robotic applications such as spraying, welding, and additive manufacturing frequently require fast, accurate, and uniform motion along a 3D spatial curve. To increase process throughput, some manufacturers propose a dual-robot setup to overcome the speed limitation of a single robot. Industrial robot motion is programmed through waypoints connected by motion primitives (Cartesian linear… ▽ More Industrial robotic applications such as spraying, welding, and additive manufacturing frequently require fast, accurate, and uniform motion along a 3D spatial curve. To increase process throughput, some manufacturers propose a dual-robot setup to overcome the speed limitation of a single robot. Industrial robot motion is programmed through waypoints connected by motion primitives (Cartesian linear and circular paths and linear joint paths at constant Cartesian speed). The actual robot motion is affected by the blending between these motion primitives and the pose of the robot (an outstretched/near-singularity pose tends to have larger path tracking errors). Choosing the waypoints and the speed along each motion segment to achieve the performance requirement is challenging. At present, there is no automated solution, and laborious manual tuning by robot experts is needed to approach the desired performance. In this paper, we present a systematic three-step approach to designing and programming a dual robot system to optimize system performance. The first step is to select the relative placement between the two robots based on the specified relative motion path. The second step is to select the relative waypoints and the motion primitives. The final step is to update the waypoints iteratively based on the actual measured relative motion. Waypoint iteration is first executed in simulation and then completed using the actual robots. For performance assessment, we use the mean path speed subject to the relative position and orientation constraints and the path speed uniformity constraint. We have demonstrated the effectiveness of this method on two systems, a physical testbed of two ABB robots and a simulation testbed of two FANUC robots, for two challenging test curves. △ Less

Submitted 14 August, 2024; v1 submitted 9 April, 2024; originally announced April 2024.

Iterative Planning for Multi-agent Systems: An Application in Energy-Aware UAV-UGV Cooperative Task Site Assignments

Authors: Neelanga Thelasingha, Agung Julius, James Humann, Jean-Paul Reddinger, James Dotterweich, Marshal Childers

Abstract: This paper presents an iterative planning framework for multi-agent systems with hybrid state spaces. The framework uses transition systems to mathematically represent planning tasks and employs multiple solvers to iteratively improve the plan until computation resources are exhausted. When integrating different solvers for iterative planning, we establish theoretical guarantees on the mathematica… ▽ More This paper presents an iterative planning framework for multi-agent systems with hybrid state spaces. The framework uses transition systems to mathematically represent planning tasks and employs multiple solvers to iteratively improve the plan until computation resources are exhausted. When integrating different solvers for iterative planning, we establish theoretical guarantees on the mathematical framework to ensure recursive feasibility. The proposed framework enables continual improvement of solution optimality, efficiently using allocated computation resources. The proposed method is validated by applying it to an energy-aware UGV-UAV cooperative task site assignment. The results demonstrate the continual solution improvement while preserving real-time implementation ability compared to algorithms proposed in the literature. △ Less

Submitted 19 March, 2024; v1 submitted 16 January, 2024; originally announced January 2024.

High-Speed High-Accuracy Spatial Curve Tracking Using Motion Primitives in Industrial Robots

Authors: Honglu He, Chen-lung Lu, Yunshi Wen, Glenn Saunders, Pinghai Yang, Jeffrey Schoonover, Agung Julius, John T. Wen

Abstract: Industrial robots are increasingly deployed in applications requiring an end effector tool to closely track a specified path, such as in spraying and welding. Performance and productivity present possibly conflicting objectives: tracking accuracy, path speed, and motion uniformity. Industrial robots are programmed through motion primitives consisting of waypoints connected by pre-defined motion se… ▽ More Industrial robots are increasingly deployed in applications requiring an end effector tool to closely track a specified path, such as in spraying and welding. Performance and productivity present possibly conflicting objectives: tracking accuracy, path speed, and motion uniformity. Industrial robots are programmed through motion primitives consisting of waypoints connected by pre-defined motion segments, with specified parameters such as path speed and blending zone. The actual executed robot motion depends on the robot joint servo controller and joint motion constraints (velocity, acceleration, etc.) which are largely unknown to the users. Programming a robot to achieve the desired performance today is time-consuming and mostly manual, requiring tuning a large number of coupled parameters in the motion primitives. The performance also depends on the choice of additional parameters: possible redundant degrees of freedom, location of the target curve, and the robot configuration. This paper presents a systematic approach to optimize the robot motion primitives for performance. The approach first selects the static parameters, then the motion primitives, and finally iteratively update the waypoints to minimize the tracking error. The ultimate performance objective is to maximize the path speed subject to the tracking accuracy and speed uniformity constraints over the entire path. We have demonstrated the effectiveness of this approach in simulation for ABB and FANUC robots for two challenging example curves, and experimentally for an ABB robot. Comparing with the baseline using the current industry practice, the optimized performance shows over 200% performance improvement. △ Less

Submitted 5 January, 2023; originally announced January 2023.

Distributed Monitoring of Robot Swarms with Swarm Signal Temporal Logic

Authors: Ruixuan Yan, Agung Julius

Abstract: In this paper, we develop a distributed monitoring framework for robot swarms so that the agents can monitor whether the executions of robot swarms satisfy Swarm Signal Temporal Logic (SwarmSTL) formulas. We define generalized moments (GMs) to represent swarm features. A dynamic generalized moments consensus algorithm (GMCA) with Kalman filter (KF) is proposed so that each agent can estimate the G… ▽ More In this paper, we develop a distributed monitoring framework for robot swarms so that the agents can monitor whether the executions of robot swarms satisfy Swarm Signal Temporal Logic (SwarmSTL) formulas. We define generalized moments (GMs) to represent swarm features. A dynamic generalized moments consensus algorithm (GMCA) with Kalman filter (KF) is proposed so that each agent can estimate the GMs. Also, we obtain an upper bound for the error between an agent's estimate and the actual GMs. This bound is independent of the motion of the agents. We also propose rules for monitoring SwarmSTL temporal and logical operators. As a result, the agents can monitor whether the swarm satisfies SwarmSTL formulas with a certain confidence level using these rules and the bound of the estimation error. The distributed monitoring framework is applied to a swarm transporting supplies example, where we also show the efficacy of the Kalman filter in the dynamic generalized moments consensus process. △ Less

Submitted 2 May, 2020; v1 submitted 26 April, 2020; originally announced April 2020.

Temporal Logic Inference for Hybrid System Observation with Spatial and Temporal Uncertainties

Authors: Zhe Xu, Yi Deng, Agung Julius

Abstract: In this paper, we present a mechanism for building hybrid system observers to differentiate between specific positions of the hybrid system. The mechanism is designed through inferring metric temporal logic (MTL) formulae from simulated trajectories from the hybrid system. We first approximate the system behavior by simulating finitely many trajectories with timerobust tube segments around them. T… ▽ More In this paper, we present a mechanism for building hybrid system observers to differentiate between specific positions of the hybrid system. The mechanism is designed through inferring metric temporal logic (MTL) formulae from simulated trajectories from the hybrid system. We first approximate the system behavior by simulating finitely many trajectories with timerobust tube segments around them. These time-robust tube segments account for both spatial and temporal uncertainties that exist in the hybrid system with initial state variations. The inferred MTL formulae classify different time-robust tube segments and thus can be used for classifying the hybrid system behaviors in a provably correct fashion. We implement our approach on a model of a smart building testbed to distinguish two cases of room occupancy. △ Less

Submitted 4 April, 2020; originally announced April 2020.

Controller Synthesis of Wind Turbine Generator and Energy Storage System with Stochastic Wind Variations under Temporal Logic Specifications

Authors: Zhe Xu, Agung Julius, Ufuk Topcu, Joe H. Chow

Abstract: In this paper, we present a controller synthesis approach for wind turbine generators (WTG) and energy storage systems with metric temporal logic (MTL) specifications, with provable probabilistic guarantees in the stochastic environment of wind power generation. The MTL specifications are requirements for the grid frequency deviations, WTG rotor speed variations and the power flow constraints at d… ▽ More In this paper, we present a controller synthesis approach for wind turbine generators (WTG) and energy storage systems with metric temporal logic (MTL) specifications, with provable probabilistic guarantees in the stochastic environment of wind power generation. The MTL specifications are requirements for the grid frequency deviations, WTG rotor speed variations and the power flow constraints at different lines. We present the stochastic control bisimulation function, which bounds the divergence of the trajectories of a switched stochastic control system and the switched nominal control system in a probabilistic fashion.We first design a feedforward controller by solving an optimization problem for the nominal trajectory of the deterministic control system with robustness against initial state variations and stochastic uncertainties. Then we generate a feedback control law from the data of the simulated trajectories. We implement our control method on both a four-bus system and a nine-bus system, and test the effectiveness of the method with a generation loss disturbance. We also test the advantage of the feedback controller over the feedforward controller when unexpected disturbance occurs. △ Less

Submitted 26 November, 2019; originally announced November 2019.

Continuous Circadian Phase Estimation Using Adaptive Notch Filter

Authors: Wei Qiao, Kyle Altman, Agung Julius, Bernard Possidente, John T. Wen

Abstract: Actigraphy has been widely used for the analysis of circadian rhythm. Current practice applies regression analysis to data from multiple days to estimate the circadian phase. This paper presents a filtering method for online processing of biometric data to estimate the circadian phase. We apply the proposed method on actigraphy data of fruit flies (Drosophila melanogaster). Actigraphy has been widely used for the analysis of circadian rhythm. Current practice applies regression analysis to data from multiple days to estimate the circadian phase. This paper presents a filtering method for online processing of biometric data to estimate the circadian phase. We apply the proposed method on actigraphy data of fruit flies (Drosophila melanogaster). △ Less

Submitted 30 March, 2018; originally announced April 2018.

An MILP Approach for Real-time Optimal Controller Synthesis with Metric Temporal Logic Specifications

Authors: Sayan Saha, A. Agung Julius

Abstract: The fundamental idea of this work is to synthesize reactive controllers such that closed-loop execution trajectories of the system satisfy desired specifications that ensure correct system behaviors, while optimizing a desired performance criteria. In our approach, the correctness of a system's behavior can be defined according to the system's relation to the environment, for example, the output t… ▽ More The fundamental idea of this work is to synthesize reactive controllers such that closed-loop execution trajectories of the system satisfy desired specifications that ensure correct system behaviors, while optimizing a desired performance criteria. In our approach, the correctness of a system's behavior can be defined according to the system's relation to the environment, for example, the output trajectories of the system terminate in a goal set without entering an unsafe set. Using Metric Temporal Logic (MTL) specifications we can further capture complex system behaviors and timing requirements, such as the output trajectories must pass through a number of way-points within a certain time frame before terminating in the goal set. Given a Mixed Logical Dynamical (MLD) system and system specifications in terms of MTL formula or simpler reach-avoid specifications, our goal is to find a closed-loop trajectory that satisfies the specifications, in non-deterministic environments. Using an MILP framework we search over the space of input signals to obtain such valid trajectories of the system, by adding constraints to satisfy the MTL formula only when necessary, to avoid the exponential complexity of solving MILP problems. We also present experimental results for planning a path for a mobile robot through a dynamically changing environment with a desired task specification. △ Less

Submitted 8 March, 2016; originally announced March 2016.

Comments: American Control Conference 2016 (extended version)

Safe Neighborhood Computation for Hybrid System Verification

Authors: Yi Deng, Agung Julius

Abstract: For the design and implementation of engineering systems, performing model-based analysis can disclose potential safety issues at an early stage. The analysis of hybrid system models is in general difficult due to the intrinsic complexity of hybrid dynamics. In this paper, a simulation-based approach to formal verification of hybrid systems is presented. For the design and implementation of engineering systems, performing model-based analysis can disclose potential safety issues at an early stage. The analysis of hybrid system models is in general difficult due to the intrinsic complexity of hybrid dynamics. In this paper, a simulation-based approach to formal verification of hybrid systems is presented. △ Less

Submitted 22 January, 2015; originally announced January 2015.

Comments: In Proceedings HAS 2014, arXiv:1501.05405

Journal ref: EPTCS 174, 2015, pp. 1-12

Tracking Tetrahymena Pyriformis Cells using Decision Trees

Authors: Quan Wang, Yan Ou, A. Agung Julius, Kim L. Boyer, Min Jun Kim

Abstract: Matching cells over time has long been the most difficult step in cell tracking. In this paper, we approach this problem by recasting it as a classification problem. We construct a feature set for each cell, and compute a feature difference vector between a cell in the current frame and a cell in a previous frame. Then we determine whether the two cells represent the same cell over time by trainin… ▽ More Matching cells over time has long been the most difficult step in cell tracking. In this paper, we approach this problem by recasting it as a classification problem. We construct a feature set for each cell, and compute a feature difference vector between a cell in the current frame and a cell in a previous frame. Then we determine whether the two cells represent the same cell over time by training decision trees as our binary classifiers. With the output of decision trees, we are able to formulate an assignment problem for our cell association task and solve it using a modified version of the Hungarian algorithm. △ Less

Submitted 12 July, 2012; originally announced July 2012.

Comments: 21st International Conference on Pattern Recognition, 2012