-
Towards Confidence-guided Shape Completion for Robotic Applications
Authors:
Andrea Rosasco,
Stefano Berti,
Fabrizio Bottarel,
Michele Colledanchise,
Lorenzo Natale
Abstract:
Many robotic tasks involving some form of 3D visual perception greatly benefit from a complete knowledge of the working environment. However, robots often have to tackle unstructured environments and their onboard visual sensors can only provide incomplete information due to limited workspaces, clutter or object self-occlusion. In recent years, deep learning architectures for shape completion have…
▽ More
Many robotic tasks involving some form of 3D visual perception greatly benefit from a complete knowledge of the working environment. However, robots often have to tackle unstructured environments and their onboard visual sensors can only provide incomplete information due to limited workspaces, clutter or object self-occlusion. In recent years, deep learning architectures for shape completion have begun taking traction as effective means of inferring a complete 3D object representation from partial visual data. Nevertheless, most of the existing state-of-the-art approaches provide a fixed output resolution in the form of voxel grids, strictly related to the size of the neural network output stage. While this is enough for some tasks, e.g. obstacle avoidance in navigation, grasping and manipulation require finer resolutions and simply scaling up the neural network outputs is computationally expensive. In this paper, we address this limitation by proposing an object shape completion method based on an implicit 3D representation providing a confidence value for each reconstructed point. As a second contribution, we propose a gradient-based method for efficiently sampling such implicit function at an arbitrary resolution, tunable at inference time. We experimentally validate our approach by comparing reconstructed shapes with ground truths, and by deploying our shape completion algorithm in a robotic grasping pipeline. In both cases, we compare results with a state-of-the-art shape completion approach.
△ Less
Submitted 9 September, 2022;
originally announced September 2022.
-
One-Shot Open-Set Skeleton-Based Action Recognition
Authors:
Stefano Berti,
Andrea Rosasco,
Michele Colledanchise,
Lorenzo Natale
Abstract:
Action recognition is a fundamental capability for humanoid robots to interact and cooperate with humans. This application requires the action recognition system to be designed so that new actions can be easily added, while unknown actions are identified and ignored. In recent years, deep-learning approaches represented the principal solution to the Action Recognition problem. However, most models…
▽ More
Action recognition is a fundamental capability for humanoid robots to interact and cooperate with humans. This application requires the action recognition system to be designed so that new actions can be easily added, while unknown actions are identified and ignored. In recent years, deep-learning approaches represented the principal solution to the Action Recognition problem. However, most models often require a large dataset of manually-labeled samples. In this work we target One-Shot deep-learning models, because they can deal with just a single instance for class. Unfortunately, One-Shot models assume that, at inference time, the action to recognize falls into the support set and they fail when the action lies outside the support set. Few-Shot Open-Set Recognition (FSOSR) solutions attempt to address that flaw, but current solutions consider only static images and not sequences of images. Static images remain insufficient to discriminate actions such as sitting-down and standing-up. In this paper we propose a novel model that addresses the FSOSR problem with a One-Shot model that is augmented with a discriminator that rejects unknown actions. This model is useful for applications in humanoid robotics, because it allows to easily add new classes and determine whether an input sequence is among the ones that are known to the system. We show how to train the whole model in an end-to-end fashion and we perform quantitative and qualitative analyses. Finally, we provide real-world examples.
△ Less
Submitted 9 September, 2022;
originally announced September 2022.
-
Handling Concurrency in Behavior Trees
Authors:
Michele Colledanchise,
Lorenzo Natale
Abstract:
This paper addresses the concurrency issues affecting Behavior Trees (BTs), a popular tool to model the behaviors of autonomous agents in the video game and the robotics industry.
BT designers can easily build complex behaviors composing simpler ones, which represents a key advantage of BTs. The parallel composition of BTs expresses a way to combine concurrent behaviors that has high potential,…
▽ More
This paper addresses the concurrency issues affecting Behavior Trees (BTs), a popular tool to model the behaviors of autonomous agents in the video game and the robotics industry.
BT designers can easily build complex behaviors composing simpler ones, which represents a key advantage of BTs. The parallel composition of BTs expresses a way to combine concurrent behaviors that has high potential, since composing pre-existing BTs in parallel results easier than composing in parallel classical control architectures, as finite state machines or teleo-reactive programs. However, BT designers rarely use such composition due to the underlying concurrency problems similar to the ones faced in concurrent programming. As a result, the parallel composition, despite its potential, finds application only in the composition of simple behaviors or where the designer can guarantee the absence of conflicts by design.
In this paper, we define two new BT nodes to tackle the concurrency problems in BTs and we show how to exploit them to create predictable behaviors. In addition, we introduce measures to assess execution performance and show how different design choices affect them. We validate our approach in both simulations and the real world. Simulated experiments provide statistically significant data, whereas real-world experiments show the applicability of our method on real robots. We provided an open-source implementation of the novel BT formulation and published all the source code to reproduce the numerical examples and experiments.
△ Less
Submitted 22 October, 2021;
originally announced October 2021.
-
Active Perception for Ambiguous Objects Classification
Authors:
Evgenii Safronov,
Nicola Piga,
Michele Colledanchise,
Lorenzo Natale
Abstract:
Recent visual pose estimation and tracking solutions provide notable results on popular datasets such as T-LESS and YCB. However, in the real world, we can find ambiguous objects that do not allow exact classification and detection from a single view. In this work, we propose a framework that, given a single view of an object, provides the coordinates of a next viewpoint to discriminate the object…
▽ More
Recent visual pose estimation and tracking solutions provide notable results on popular datasets such as T-LESS and YCB. However, in the real world, we can find ambiguous objects that do not allow exact classification and detection from a single view. In this work, we propose a framework that, given a single view of an object, provides the coordinates of a next viewpoint to discriminate the object against similar ones, if any, and eliminates ambiguities. We also describe a complete pipeline from a real object's scans to the viewpoint selection and classification. We validate our approach with a Franka Emika Panda robot and common household objects featured with ambiguities. We released the source code to reproduce our experiments.
△ Less
Submitted 2 August, 2021;
originally announced August 2021.
-
On the Implementation of Behavior Trees in Robotics
Authors:
Michele Colledanchise,
Lorenzo Natale
Abstract:
There is a growing interest in Behavior Trees (BTs) as a tool to describe and implement robot behaviors. BTs were devised in the video game industry and their adoption in robotics resulted in the development of ad-hoc libraries to design and execute BTs that fit complex robotics software architectures.
While there is broad consensus on how BTs work, some characteristics rely on the implementatio…
▽ More
There is a growing interest in Behavior Trees (BTs) as a tool to describe and implement robot behaviors. BTs were devised in the video game industry and their adoption in robotics resulted in the development of ad-hoc libraries to design and execute BTs that fit complex robotics software architectures.
While there is broad consensus on how BTs work, some characteristics rely on the implementation choices done in the specific software library used.
In this letter, we outline practical aspects in the adoption of BTs and the solutions devised by the robotics community to fully exploit the advantages of BTs in real robots. We also overview the solutions proposed in open-source libraries used in robotics, we show how BTs fit in robotic software architecture, and we present a use case example.
△ Less
Submitted 29 June, 2021;
originally announced June 2021.
-
A Toolchain to Design, Execute, and Monitor Robots Behaviors
Authors:
Michele Colledanchise,
Giuseppe Cicala,
Daniele E. Domenichelli,
Lorenzo Natale,
Armando Tacchella
Abstract:
In this paper, we present a toolchain to design, execute, and verify robot behaviors. The toolchain follows the guidelines defined by the EU H2020 project RobMoSys and encodes the robot deliberation as a Behavior Tree (BT), a directed tree where the internal nodes model behavior composition and leaf nodes model action or measurement operations. Such leaf nodes take the form of a statechart (SC), w…
▽ More
In this paper, we present a toolchain to design, execute, and verify robot behaviors. The toolchain follows the guidelines defined by the EU H2020 project RobMoSys and encodes the robot deliberation as a Behavior Tree (BT), a directed tree where the internal nodes model behavior composition and leaf nodes model action or measurement operations. Such leaf nodes take the form of a statechart (SC), which runs in separate threads, whose states perform basic arithmetic operations and send commands to the robot. The toolchain provides the ability to define a runtime monitor for a given system specification that warns the user whenever a given specification is violated.
We validated the toolchain in a simulated experiment that we made reproducible in an OS-virtualization environment.
△ Less
Submitted 29 June, 2021;
originally announced June 2021.
-
Formalizing the Execution Context of Behavior Trees for Runtime Verification of Deliberative Policies
Authors:
Michele Colledanchise,
Giuseppe Cicala,
Daniele E. Domenichelli,
Lorenzo Natale,
Armando Tacchella
Abstract:
In this paper, we enable automated property verification of deliberative components in robot control architectures. We focus on formalizing the execution context of Behavior Trees (BTs) to provide a scalable, yet formally grounded, methodology to enable runtime verification and prevent unexpected robot behaviors. To this end, we consider a message-passing model that accommodates both synchronous a…
▽ More
In this paper, we enable automated property verification of deliberative components in robot control architectures. We focus on formalizing the execution context of Behavior Trees (BTs) to provide a scalable, yet formally grounded, methodology to enable runtime verification and prevent unexpected robot behaviors. To this end, we consider a message-passing model that accommodates both synchronous and asynchronous composition of parallel components, in which BTs and other components execute and interact according to the communication patterns commonly adopted in robotic software architectures. We introduce a formal property specification language to encode requirements and build runtime monitors. We performed a set of experiments, both on simulations and on the real robot, demonstrating the feasibility of our approach in a realistic application and its integration in a typical robot software architecture. We also provide an OS-level virtualization environment to reproduce the experiments in the simulated scenario.
△ Less
Submitted 16 September, 2021; v1 submitted 23 June, 2021;
originally announced June 2021.
-
Address Behaviour Vulnerabilities in the Next Generation of Autonomous Robots
Authors:
Michele Colledanchise
Abstract:
Robots applications in our daily life increase at an unprecedented pace. As robots will soon operate "out in the wild", we must identify the safety and security vulnerabilities they will face. Robotics researchers and manufacturers focus their attention on new, cheaper, and more reliable applications. Still, they often disregard the operability in adversarial environments where a trusted or untrus…
▽ More
Robots applications in our daily life increase at an unprecedented pace. As robots will soon operate "out in the wild", we must identify the safety and security vulnerabilities they will face. Robotics researchers and manufacturers focus their attention on new, cheaper, and more reliable applications. Still, they often disregard the operability in adversarial environments where a trusted or untrusted user can jeopardize or even alter the robot's task.
In this paper, we identify a new paradigm of security threats in the next generation of robots. These threats fall beyond the known hardware or network-based ones, and we must find new solutions to address them. These new threats include malicious use of the robot's privileged access, tampering with the robot sensors system, and tricking the robot's deliberation into harmful behaviors. We provide a taxonomy of attacks that exploit these vulnerabilities with realistic examples, and we outline effective countermeasures to prevent better, detect, and mitigate them.
△ Less
Submitted 17 November, 2021; v1 submitted 24 March, 2021;
originally announced March 2021.
-
Compact Belief State Representation for Task Planning
Authors:
Evgenii Safronov,
Michele Colledanchise,
Lorenzo Natale
Abstract:
Task planning in a probabilistic belief state domains allows generating complex and robust execution policies in those domains affected by state uncertainty. The performance of a task planner relies on the belief state representation. However, current belief state representation becomes easily intractable as the number of variables and execution time grows. To address this problem, we developed a…
▽ More
Task planning in a probabilistic belief state domains allows generating complex and robust execution policies in those domains affected by state uncertainty. The performance of a task planner relies on the belief state representation. However, current belief state representation becomes easily intractable as the number of variables and execution time grows. To address this problem, we developed a novel belief state representation based on cartesian product and union operations over belief substates. These two operations and single variable assignment nodes form And-Or directed acyclic graph of Belief State (AOBS). We show how to apply actions with probabilistic outcomes and measure the probability of conditions holding over belief state. We evaluated AOBS performance in simulated forward state space exploration. We compared the size of AOBS with the size of Binary Decision Diagrams (BDD) that were previously used to represent belief state. We show that AOBS representation is not only much more compact than a full belief state but it also scales better than BDD for most of the cases.
△ Less
Submitted 21 August, 2020;
originally announced August 2020.
-
Task Planning with Belief Behavior Trees
Authors:
Evgenii Safronov,
Michele Colledanchise,
Lorenzo Natale
Abstract:
In this paper, we propose Belief Behavior Trees (BBTs), an extension to Behavior Trees (BTs) that allows to automatically create a policy that controls a robot in partially observable environments. We extend the semantic of BTs to account for the uncertainty that affects both the conditions and action nodes of the BT. The tree gets synthesized following a planning strategy for BTs proposed recentl…
▽ More
In this paper, we propose Belief Behavior Trees (BBTs), an extension to Behavior Trees (BTs) that allows to automatically create a policy that controls a robot in partially observable environments. We extend the semantic of BTs to account for the uncertainty that affects both the conditions and action nodes of the BT. The tree gets synthesized following a planning strategy for BTs proposed recently: from a set of goal conditions we iteratively select a goal and find the action, or in general the subtree, that satisfies it. Such action may have preconditions that do not hold. For those preconditions, we find an action or subtree in the same fashion. We extend this approach by including, in the planner, actions that have the purpose to reduce the uncertainty that affects the value of a condition node in the BT (for example, turning on the lights to have better lighting conditions). We demonstrate that BBTs allows task planning with non-deterministic outcomes for actions. We provide experimental validation of our approach in a real robotic scenario and - for sake of reproducibility - in a simulated one.
△ Less
Submitted 21 August, 2020;
originally announced August 2020.
-
Act, Perceive, and Plan in Belief Space for Robot Localization
Authors:
Michele Colledanchise,
Damiano Malafronte,
Lorenzo Natale
Abstract:
In this paper, we outline an interleaved acting and planning technique to rapidly reduce the uncertainty of the estimated robot's pose by perceiving relevant information from the environment, as recognizing an object or asking someone for a direction.
Generally, existing localization approaches rely on low-level geometric features such as points, lines, and planes, while these approaches provide…
▽ More
In this paper, we outline an interleaved acting and planning technique to rapidly reduce the uncertainty of the estimated robot's pose by perceiving relevant information from the environment, as recognizing an object or asking someone for a direction.
Generally, existing localization approaches rely on low-level geometric features such as points, lines, and planes, while these approaches provide the desired accuracy, they may require time to converge, especially with incorrect initial guesses. In our approach, a task planner computes a sequence of action and perception tasks to actively obtain relevant information from the robot's perception system. We validate our approach in large state spaces, to show how the approach scales, and in real environments, to show the applicability of our method on real robots.
We prove that our approach is sound, probabilistically complete, and tractable in practical cases.
△ Less
Submitted 21 February, 2020; v1 submitted 19 February, 2020;
originally announced February 2020.
-
Analysis and Exploitation of Synchronized Parallel Executions in Behavior Trees
Authors:
Michele Colledanchise,
Lorenzo Natale
Abstract:
Behavior Trees (BTs) are becoming a popular tool to model the behaviors of autonomous agents in the computer game and the robotics industry. One of the key advantages of BTs lies in their composability, where complex behaviors can be built by composing simpler ones. The parallel composition is the one with the highest potential since the complexity of composing pre-existing behaviors in parallel i…
▽ More
Behavior Trees (BTs) are becoming a popular tool to model the behaviors of autonomous agents in the computer game and the robotics industry. One of the key advantages of BTs lies in their composability, where complex behaviors can be built by composing simpler ones. The parallel composition is the one with the highest potential since the complexity of composing pre-existing behaviors in parallel is much lower than the one needed using classical control architectures as finite state machines. However, the parallel composition is rarely used due to the underlying concurrency problems that are similar to the ones faced in concurrent programming. In this paper, we define two synchronization techniques to tackle the concurrency problems in BTs compositions and we show how to exploit them to improve behavior predictability. Also, we introduce measures to assess execution performance, and we show how design choices can affect them. To illustrate the proposed framework, we provide a set of experiments using the R1 robot and we gather statistically-significant data.
△ Less
Submitted 5 August, 2019;
originally announced August 2019.
-
Improving the Parallel Execution of Behavior Trees
Authors:
Michele Colledanchise,
Lorenzo Natale
Abstract:
Behavior Trees (BTs) have become a popular framework for designing controllers of autonomous agents in the computer game and in the robotics industry. One of the key advantages of BTs lies in their modularity, where independent modules can be composed to create more complex ones. In the classical formulation of BTs, modules can be composed using one of the three operators: Sequence, Fallback, and…
▽ More
Behavior Trees (BTs) have become a popular framework for designing controllers of autonomous agents in the computer game and in the robotics industry. One of the key advantages of BTs lies in their modularity, where independent modules can be composed to create more complex ones. In the classical formulation of BTs, modules can be composed using one of the three operators: Sequence, Fallback, and Parallel. The Parallel operator is rarely used despite its strong potential against other control architectures as Finite State Machines. This is due to the fact that concurrent actions may lead to unexpected problems similar to the ones experienced in concurrent programming. In this paper, we introduce Concurrent BTs (CBTs) as a generalization of BTs in which we introduce the notions of progress and resource usage. We show how CBTs allow safe concurrent executions of actions and we analyze the approach from a mathematical standpoint. To illustrate the use of CBTs, we provide a set of use cases in robotics scenarios.
△ Less
Submitted 13 September, 2018;
originally announced September 2018.
-
Behavior Trees in Robotics and AI: An Introduction
Authors:
Michele Colledanchise,
Petter Ögren
Abstract:
A Behavior Tree (BT) is a way to structure the switching between different tasks in an autonomous agent, such as a robot or a virtual entity in a computer game. BTs are a very efficient way of creating complex systems that are both modular and reactive. These properties are crucial in many applications, which has led to the spread of BT from computer game programming to many branches of AI and Rob…
▽ More
A Behavior Tree (BT) is a way to structure the switching between different tasks in an autonomous agent, such as a robot or a virtual entity in a computer game. BTs are a very efficient way of creating complex systems that are both modular and reactive. These properties are crucial in many applications, which has led to the spread of BT from computer game programming to many branches of AI and Robotics. In this book, we will first give an introduction to BTs, then we describe how BTs relate to, and in many cases generalize, earlier switching structures. These ideas are then used as a foundation for a set of efficient and easy to use design principles. Properties such as safety, robustness, and efficiency are important for an autonomous system, and we describe a set of tools for formally analyzing these using a state space description of BTs. With the new analysis tools, we can formalize the descriptions of how BTs generalize earlier approaches. We also show the use of BTs in automated planning and machine learning. Finally, we describe an extended set of tools to capture the behavior of Stochastic BTs, where the outcomes of actions are described by probabilities. These tools enable the computation of both success probabilities and time to completion.
△ Less
Submitted 25 October, 2022; v1 submitted 31 August, 2017;
originally announced September 2017.
-
Towards Blended Reactive Planning and Acting using Behavior Trees
Authors:
Michele Colledanchise,
Diogo Almeida,
Petter Ögren
Abstract:
In this paper, we show how a planning algorithm can be used to automatically create and update a Behavior Tree (BT), controlling a robot in a dynamic environment. The planning part of the algorithm is based on the idea of back chaining. Starting from a goal condition we iteratively select actions to achieve that goal, and if those actions have unmet preconditions, they are extended with actions to…
▽ More
In this paper, we show how a planning algorithm can be used to automatically create and update a Behavior Tree (BT), controlling a robot in a dynamic environment. The planning part of the algorithm is based on the idea of back chaining. Starting from a goal condition we iteratively select actions to achieve that goal, and if those actions have unmet preconditions, they are extended with actions to achieve them in the same way. The fact that BTs are inherently modular and reactive makes the proposed solution blend acting and planning in a way that enables the robot to efficiently react to external disturbances. If an external agent undoes an action the robot reexecutes it without re-planning, and if an external agent helps the robot, it skips the corresponding actions, again without replanning. We illustrate our approach in two different robotics scenarios.
△ Less
Submitted 14 September, 2018; v1 submitted 1 November, 2016;
originally announced November 2016.
-
Learning of Behavior Trees for Autonomous Agents
Authors:
Michele Colledanchise,
Ramviyas Parasuraman,
Petter Ögren
Abstract:
Definition of an accurate system model for Automated Planner (AP) is often impractical, especially for real-world problems. Conversely, off-the-shelf planners fail to scale up and are domain dependent. These drawbacks are inherited from conventional transition systems such as Finite State Machines (FSMs) that describes the action-plan execution generated by the AP. On the other hand, Behavior Tree…
▽ More
Definition of an accurate system model for Automated Planner (AP) is often impractical, especially for real-world problems. Conversely, off-the-shelf planners fail to scale up and are domain dependent. These drawbacks are inherited from conventional transition systems such as Finite State Machines (FSMs) that describes the action-plan execution generated by the AP. On the other hand, Behavior Trees (BTs) represent a valid alternative to FSMs presenting many advantages in terms of modularity, reactiveness, scalability and domain-independence. In this paper, we propose a model-free AP framework using Genetic Programming (GP) to derive an optimal BT for an autonomous agent to achieve a given goal in unknown (but fully observable) environments. We illustrate the proposed framework using experiments conducted with an open source benchmark Mario AI for automated generation of BTs that can play the game character Mario to complete a certain level at various levels of difficulty to include enemies and obstacles.
△ Less
Submitted 22 April, 2015;
originally announced April 2015.
-
Adaptive Fault Tolerant Execution of Multi-Robot Missions using Behavior Trees
Authors:
Michele Colledanchise,
Alejandro Marzinotto,
Dimos V. Dimarogonas,
Petter Ögren
Abstract:
Multi-robot teams offer possibilities of improved performance and fault tolerance, compared to single robot solutions. In this paper, we show how to realize those possibilities when starting from a single robot system controlled by a Behavior Tree (BT). By extending the single robot BT to a multi-robot BT, we are able to combine the fault tolerant properties of the BT, in terms of built-in fallbac…
▽ More
Multi-robot teams offer possibilities of improved performance and fault tolerance, compared to single robot solutions. In this paper, we show how to realize those possibilities when starting from a single robot system controlled by a Behavior Tree (BT). By extending the single robot BT to a multi-robot BT, we are able to combine the fault tolerant properties of the BT, in terms of built-in fallbacks, with the fault tolerance inherent in multi-robot approaches, in terms of a faulty robot being replaced by another one. Furthermore, we improve performance by identifying and taking advantage of the opportunities of parallel task execution, that are present in the single robot BT. Analyzing the proposed approach, we present results regarding how mission performance is affected by minor faults (a robot losing one capability) as well as major faults (a robot losing all its capabilities). Finally, a detailed example is provided to illustrate the approach.
△ Less
Submitted 10 February, 2015;
originally announced February 2015.