Showing 1–1 of 1 results for author: Dimick, N J

Situated Multimodal Control of a Mobile Robot: Navigation through a Virtual Environment

Authors: Katherine Krajovic, Nikhil Krishnaswamy, Nathaniel J. Dimick, R. Pito Salas, James Pustejovsky

Abstract: We present a new interface for controlling a navigation robot in novel environments using coordinated gesture and language. We use a TurtleBot3 robot with a LIDAR and a camera, an embodied simulation of what the robot has encountered while exploring, and a cross-platform bridge facilitating generic communication. A human partner can deliver instructions to the robot using spoken English and gestur… ▽ More We present a new interface for controlling a navigation robot in novel environments using coordinated gesture and language. We use a TurtleBot3 robot with a LIDAR and a camera, an embodied simulation of what the robot has encountered while exploring, and a cross-platform bridge facilitating generic communication. A human partner can deliver instructions to the robot using spoken English and gestures relative to the simulated environment, to guide the robot through navigation tasks. △ Less

Submitted 13 July, 2020; originally announced July 2020.

Comments: 4 pages, 1 table, 4 figures, proceedings of RoboDIAL special session a SigDIAL 2020