-
RoboEval: Where Robotic Manipulation Meets Structured and Scalable Evaluation
Authors:
Yi Ru Wang,
Carter Ung,
Grant Tannert,
Jiafei Duan,
Josephine Li,
Amy Le,
Rishabh Oswal,
Markus Grotz,
Wilbert Pumacay,
Yuquan Deng,
Ranjay Krishna,
Dieter Fox,
Siddhartha Srinivasa
Abstract:
We present RoboEval, a simulation benchmark and structured evaluation framework designed to reveal the limitations of current bimanual manipulation policies. While prior benchmarks report only binary task success, we show that such metrics often conceal critical weaknesses in policy behavior -- such as poor coordination, slipping during grasping, or asymmetric arm usage. RoboEval introduces a suit…
▽ More
We present RoboEval, a simulation benchmark and structured evaluation framework designed to reveal the limitations of current bimanual manipulation policies. While prior benchmarks report only binary task success, we show that such metrics often conceal critical weaknesses in policy behavior -- such as poor coordination, slipping during grasping, or asymmetric arm usage. RoboEval introduces a suite of tiered, semantically grounded tasks decomposed into skill-specific stages, with variations that systematically challenge spatial, physical, and coordination capabilities. Tasks are paired with fine-grained diagnostic metrics and 3000+ human demonstrations to support imitation learning. Our experiments reveal that policies with similar success rates diverge in how tasks are executed -- some struggle with alignment, others with temporally consistent bimanual control. We find that behavioral metrics correlate with success in over half of task-metric pairs, and remain informative even when binary success saturates. By pinpointing when and how policies fail, RoboEval enables a deeper, more actionable understanding of robotic manipulation -- and highlights the need for evaluation tools that go beyond success alone.
△ Less
Submitted 1 July, 2025;
originally announced July 2025.
-
Six Feet Apart: Online Payments During the COVID-19 Pandemic
Authors:
Omar Shaikh,
Cassandra Ung,
Diyi Yang,
Felipe Chacon
Abstract:
Since the COVID-19 pandemic, businesses have faced unprecedented challenges when trying to remain open. Because COVID-19 spreads through aerosolized droplets, businesses were forced to distance their services; in some cases, distancing may have involved moving business services online. In this work, we explore digitization strategies used by small businesses that remained open during the pandemic,…
▽ More
Since the COVID-19 pandemic, businesses have faced unprecedented challenges when trying to remain open. Because COVID-19 spreads through aerosolized droplets, businesses were forced to distance their services; in some cases, distancing may have involved moving business services online. In this work, we explore digitization strategies used by small businesses that remained open during the pandemic, and survey/interview small businesses owners to understand preliminary challenges associated with moving online. Furthermore, we analyze payments from 400K businesses across Japan, Australia, United States, Great Britain, and Canada. Following initial government interventions, we observe (at minimum for each country) a 47% increase in digitizing businesses compared to pre-pandemic levels, with about 80% of surveyed businesses digitizing in under a week. From both our quantitative models and our surveys/interviews, we find that businesses rapidly digitized at the start of the pandemic in preparation of future uncertainty. We also conduct a case-study of initial digitization in the United States, examining finer relationships between specific government interventions, business sectors, political orientation, and resulting digitization shifts. Finally, we discuss the implications of rapid & widespread digitization for small businesses in the context of usability challenges and interpersonal interactions, while highlighting potential shifts in pre-existing social norms.
△ Less
Submitted 29 April, 2022;
originally announced April 2022.
-
On the strong chromatic index and maximum induced matching of tree-cographs and permutation graphs
Authors:
Ton Kloks,
Chin-Ting Ung,
Yue-Li Wang
Abstract:
We show that there exist linear-time algorithms that compute the strong chromatic index and a maximum induced matching of tree-cographs when the decomposition tree is a part of the input. We also show that there exists an efficient algorithm for the strong chromatic index of permutation graphs.
We show that there exist linear-time algorithms that compute the strong chromatic index and a maximum induced matching of tree-cographs when the decomposition tree is a part of the input. We also show that there exists an efficient algorithm for the strong chromatic index of permutation graphs.
△ Less
Submitted 8 October, 2011;
originally announced October 2011.
-
Algorithms for the strong chromatic index of Halin graphs, distance-hereditary graphs and maximal outerplanar graphs
Authors:
Ton Kloks,
Sheung-Hung Poon,
Chin-Ting Ung,
Yue-Li Wang
Abstract:
We show that there exist linear-time algorithms that compute the strong chromatic index of Halin graphs, of maximal outerplanar graphs and of distance-hereditary graphs.
We show that there exist linear-time algorithms that compute the strong chromatic index of Halin graphs, of maximal outerplanar graphs and of distance-hereditary graphs.
△ Less
Submitted 4 October, 2011;
originally announced October 2011.
