-
Monocular Differentiable Rendering for Self-Supervised 3D Object Detection
Authors:
Deniz Beker,
Hiroharu Kato,
Mihai Adrian Morariu,
Takahiro Ando,
Toru Matsuoka,
Wadim Kehl,
Adrien Gaidon
Abstract:
3D object detection from monocular images is an ill-posed problem due to the projective entanglement of depth and scale. To overcome this ambiguity, we present a novel self-supervised method for textured 3D shape reconstruction and pose estimation of rigid objects with the help of strong shape priors and 2D instance masks. Our method predicts the 3D location and meshes of each object in an image u…
▽ More
3D object detection from monocular images is an ill-posed problem due to the projective entanglement of depth and scale. To overcome this ambiguity, we present a novel self-supervised method for textured 3D shape reconstruction and pose estimation of rigid objects with the help of strong shape priors and 2D instance masks. Our method predicts the 3D location and meshes of each object in an image using differentiable rendering and a self-supervised objective derived from a pretrained monocular depth estimation network. We use the KITTI 3D object detection dataset to evaluate the accuracy of the method. Experiments demonstrate that we can effectively use noisy monocular depth and differentiable rendering as an alternative to expensive 3D ground-truth labels or LiDAR information.
△ Less
Submitted 30 September, 2020;
originally announced September 2020.
-
Differentiable Rendering: A Survey
Authors:
Hiroharu Kato,
Deniz Beker,
Mihai Morariu,
Takahiro Ando,
Toru Matsuoka,
Wadim Kehl,
Adrien Gaidon
Abstract:
Deep neural networks (DNNs) have shown remarkable performance improvements on vision-related tasks such as object detection or image segmentation. Despite their success, they generally lack the understanding of 3D objects which form the image, as it is not always possible to collect 3D information about the scene or to easily annotate it. Differentiable rendering is a novel field which allows the…
▽ More
Deep neural networks (DNNs) have shown remarkable performance improvements on vision-related tasks such as object detection or image segmentation. Despite their success, they generally lack the understanding of 3D objects which form the image, as it is not always possible to collect 3D information about the scene or to easily annotate it. Differentiable rendering is a novel field which allows the gradients of 3D objects to be calculated and propagated through images. It also reduces the requirement of 3D data collection and annotation, while enabling higher success rate in various applications. This paper reviews existing literature and discusses the current state of differentiable rendering, its applications and open research problems.
△ Less
Submitted 30 July, 2020; v1 submitted 22 June, 2020;
originally announced June 2020.
-
Team Delft's Robot Winner of the Amazon Picking Challenge 2016
Authors:
Carlos Hernandez,
Mukunda Bharatheesha,
Wilson Ko,
Hans Gaiser,
Jethro Tan,
Kanter van Deurzen,
Maarten de Vries,
Bas Van Mil,
Jeff van Egmond,
Ruben Burger,
Mihai Morariu,
Jihong Ju,
Xander Gerrmann,
Ronald Ensing,
Jan Van Frankenhuyzen,
Martijn Wisse
Abstract:
This paper describes Team Delft's robot, which won the Amazon Picking Challenge 2016, including both the Picking and the Stowing competitions. The goal of the challenge is to automate pick and place operations in unstructured environments, specifically the shelves in an Amazon warehouse. Team Delft's robot is based on an industrial robot arm, 3D cameras and a customized gripper. The robot's softwa…
▽ More
This paper describes Team Delft's robot, which won the Amazon Picking Challenge 2016, including both the Picking and the Stowing competitions. The goal of the challenge is to automate pick and place operations in unstructured environments, specifically the shelves in an Amazon warehouse. Team Delft's robot is based on an industrial robot arm, 3D cameras and a customized gripper. The robot's software uses ROS to integrate off-the-shelf components and modules developed specifically for the competition, implementing Deep Learning and other AI techniques for object recognition and pose estimation, grasp planning and motion planning. This paper describes the main components in the system, and discusses its performance and results at the Amazon Picking Challenge 2016 finals.
△ Less
Submitted 18 October, 2016;
originally announced October 2016.
