-
Eyes on teleporting: comparing locomotion techniques in Virtual Reality with respect to presence, sickness and spatial orientation
Authors:
Ariel Caputo,
Massimo Zancanaro,
Andrea Giachetti
Abstract:
This work compares three locomotion techniques for an immersive VR environment: two different types of teleporting (with and without animation) and a manual (joystick-based) technique. We tested the effect of these techniques on visual motion sickness, spatial awareness, presence, subjective pleasantness, and perceived difficulty of operating the navigation. We collected eye tracking and head and…
▽ More
This work compares three locomotion techniques for an immersive VR environment: two different types of teleporting (with and without animation) and a manual (joystick-based) technique. We tested the effect of these techniques on visual motion sickness, spatial awareness, presence, subjective pleasantness, and perceived difficulty of operating the navigation. We collected eye tracking and head and body orientation data to investigate the relationships between motion, vection, and sickness. Our study confirms some results already discussed in the literature regarding the reduced invasiveness and the high usability of instant teleport while increasing the evidence against the hypothesis of reduced spatial awareness induced by this technique. We reinforce the evidence about the issues of extending teleporting with animation. Furthermore, we offer some new evidence of a benefit to the user experience of the manual technique and the correlation of the sickness felt in this condition with head movements. The findings of this study contribute to the ongoing debate on the development of guidelines on navigation interfaces in specific VR environments.
△ Less
Submitted 15 December, 2023;
originally announced December 2023.
-
OO-dMVMT: A Deep Multi-view Multi-task Classification Framework for Real-time 3D Hand Gesture Classification and Segmentation
Authors:
Federico Cunico,
Federico Girella,
Andrea Avogaro,
Marco Emporio,
Andrea Giachetti,
Marco Cristani
Abstract:
Continuous mid-air hand gesture recognition based on captured hand pose streams is fundamental for human-computer interaction, particularly in AR / VR. However, many of the methods proposed to recognize heterogeneous hand gestures are tested only on the classification task, and the real-time low-latency gesture segmentation in a continuous stream is not well addressed in the literature. For this t…
▽ More
Continuous mid-air hand gesture recognition based on captured hand pose streams is fundamental for human-computer interaction, particularly in AR / VR. However, many of the methods proposed to recognize heterogeneous hand gestures are tested only on the classification task, and the real-time low-latency gesture segmentation in a continuous stream is not well addressed in the literature. For this task, we propose the On-Off deep Multi-View Multi-Task paradigm (OO-dMVMT). The idea is to exploit multiple time-local views related to hand pose and movement to generate rich gesture descriptions, along with using heterogeneous tasks to achieve high accuracy. OO-dMVMT extends the classical MVMT paradigm, where all of the multiple tasks have to be active at each time, by allowing specific tasks to switch on/off depending on whether they can apply to the input. We show that OO-dMVMT defines the new SotA on continuous/online 3D skeleton-based gesture recognition in terms of gesture classification accuracy, segmentation accuracy, false positives, and decision latency while maintaining real-time operation.
△ Less
Submitted 12 April, 2023;
originally announced April 2023.
-
SHREC 2022 Track on Online Detection of Heterogeneous Gestures
Authors:
Ariel Caputo,
Marco Emporio,
Andrea Giachetti,
Marco Cristani,
Guido Borghi,
Andrea D'Eusanio,
Minh-Quan Le,
Hai-Dang Nguyen,
Minh-Triet Tran,
F. Ambellan,
M. Hanik,
E. Nava-Yazdani,
C. von Tycowicz
Abstract:
This paper presents the outcomes of a contest organized to evaluate methods for the online recognition of heterogeneous gestures from sequences of 3D hand poses. The task is the detection of gestures belonging to a dictionary of 16 classes characterized by different pose and motion features. The dataset features continuous sequences of hand tracking data where the gestures are interleaved with non…
▽ More
This paper presents the outcomes of a contest organized to evaluate methods for the online recognition of heterogeneous gestures from sequences of 3D hand poses. The task is the detection of gestures belonging to a dictionary of 16 classes characterized by different pose and motion features. The dataset features continuous sequences of hand tracking data where the gestures are interleaved with non-significant motions. The data have been captured using the Hololens 2 finger tracking system in a realistic use-case of mixed reality interaction. The evaluation is based not only on the detection performances but also on the latency and the false positives, making it possible to understand the feasibility of practical interaction tools based on the algorithms proposed. The outcomes of the contest's evaluation demonstrate the necessity of further research to reduce recognition errors, while the computational cost of the algorithms proposed is sufficiently low.
△ Less
Submitted 22 July, 2022; v1 submitted 14 July, 2022;
originally announced July 2022.
-
Real vs Simulated Foveated Rendering to Reduce Visual Discomfort in Virtual Reality
Authors:
Ariel Caputo,
Andrea Giachetti,
Salwa Abkal,
Chiara Marchesini,
Massimo Zancanaro
Abstract:
In this paper, a study aimed at investigating the effects of real (using eye tracking to determine the fixation) and simulated foveated blurring in immersive Virtual Reality is presented. Techniques to reduce the optical flow perceived at the visual field margins are often employed in immersive Virtual Reality environments to alleviate discomfort experienced when the visual motion perception does…
▽ More
In this paper, a study aimed at investigating the effects of real (using eye tracking to determine the fixation) and simulated foveated blurring in immersive Virtual Reality is presented. Techniques to reduce the optical flow perceived at the visual field margins are often employed in immersive Virtual Reality environments to alleviate discomfort experienced when the visual motion perception does not correspond to the body's acceleration. Although still preliminary, our results suggest that for participants with higher self-declared sensitivity to sickness, there might be an improvement for nausea when using blurring. The (perceived) difficulty of the task seems to improve when the real foveated method is used.
△ Less
Submitted 4 July, 2021;
originally announced July 2021.
-
Structural biology in the clouds: The WeNMR-EOSC Ecosystem
Authors:
Rodrigo Vargas Honorato,
Panagiotis I. Koukos,
Brian Jiménez-García,
Andrei Tsaregorodtsev,
Marco Verlato,
Andrea Giachetti,
Antonio Rosato,
Alexandre M. J. J. Bonvin
Abstract:
Structural biology aims at characterizing the structural and dynamic properties of biological macromolecules at atomic details. Gaining insight into three dimensional structures of biomolecules and their interactions is critical for understanding the vast majority of cellular processes, with direct applications in health and food sciences. Since 2010, the WeNMR project (www.wenmr.eu) has implement…
▽ More
Structural biology aims at characterizing the structural and dynamic properties of biological macromolecules at atomic details. Gaining insight into three dimensional structures of biomolecules and their interactions is critical for understanding the vast majority of cellular processes, with direct applications in health and food sciences. Since 2010, the WeNMR project (www.wenmr.eu) has implemented numerous web-based services to facilitate the use of advanced computational tools by researchers in the field, using the high throughput computing infrastructure provided by EGI. These services have been further developed in subsequent initiatives under H2020 projects and are now operating as Thematic Services in the European Open Science Cloud (EOSC) portal (www.eosc-portal.eu), sending >12 millions of jobs and using around 4000 CPU-years per year. Here we review 10 years of successful e-infrastructure solutions serving a large worldwide community of over 23,000 users to date, providing them with user-friendly, web-based solutions that run complex workflows in structural biology. The current set of active WeNMR portals are described, together with the complex backend machinery that allows distributed computing resources to be harvested efficiently.
△ Less
Submitted 2 July, 2021;
originally announced July 2021.
-
SHREC 2021: Track on Skeleton-based Hand Gesture Recognition in the Wild
Authors:
Ariel Caputo,
Andrea Giachetti,
Simone Soso,
Deborah Pintani,
Andrea D'Eusanio,
Stefano Pini,
Guido Borghi,
Alessandro Simoni,
Roberto Vezzani,
Rita Cucchiara,
Andrea Ranieri,
Franca Giannini,
Katia Lupinetti,
Marina Monti,
Mehran Maghoumi,
Joseph J. LaViola Jr,
Minh-Quan Le,
Hai-Dang Nguyen,
Minh-Triet Tran
Abstract:
Gesture recognition is a fundamental tool to enable novel interaction paradigms in a variety of application scenarios like Mixed Reality environments, touchless public kiosks, entertainment systems, and more. Recognition of hand gestures can be nowadays performed directly from the stream of hand skeletons estimated by software provided by low-cost trackers (Ultraleap) and MR headsets (Hololens, Oc…
▽ More
Gesture recognition is a fundamental tool to enable novel interaction paradigms in a variety of application scenarios like Mixed Reality environments, touchless public kiosks, entertainment systems, and more. Recognition of hand gestures can be nowadays performed directly from the stream of hand skeletons estimated by software provided by low-cost trackers (Ultraleap) and MR headsets (Hololens, Oculus Quest) or by video processing software modules (e.g. Google Mediapipe). Despite the recent advancements in gesture and action recognition from skeletons, it is unclear how well the current state-of-the-art techniques can perform in a real-world scenario for the recognition of a wide set of heterogeneous gestures, as many benchmarks do not test online recognition and use limited dictionaries. This motivated the proposal of the SHREC 2021: Track on Skeleton-based Hand Gesture Recognition in the Wild. For this contest, we created a novel dataset with heterogeneous gestures featuring different types and duration. These gestures have to be found inside sequences in an online recognition scenario. This paper presents the result of the contest, showing the performances of the techniques proposed by four research groups on the challenging task compared with a simple baseline method.
△ Less
Submitted 21 June, 2021;
originally announced June 2021.
-
PAVEL: Decorative Patterns with Packed Volumetric Elements
Authors:
Filippo Andrea Fanni,
Fabio Pellacini,
Riccardo Scateni,
Andrea Giachetti
Abstract:
Many real-world hand-crafted objects are decorated with elements that are packed onto the object's surface and deformed to cover it as much as possible. Examples are artisanal ceramics and metal jewelry. Inspired by these objects, we present a method to enrich surfaces with packed volumetric decorations. Our algorithm works by first determining the locations in which to add the decorative elements…
▽ More
Many real-world hand-crafted objects are decorated with elements that are packed onto the object's surface and deformed to cover it as much as possible. Examples are artisanal ceramics and metal jewelry. Inspired by these objects, we present a method to enrich surfaces with packed volumetric decorations. Our algorithm works by first determining the locations in which to add the decorative elements and then removing the non-physical overlap between them while preserving the decoration volume. For the placement, we support several strategies depending on the desired overall motif. To remove the overlap, we use an approach based on implicit deformable models creating the qualitative effect of plastic warping while avoiding expensive and hard-to-control physical simulations. Our decorative elements can be used to enhance virtual surfaces, as well as 3D-printed pieces, by assembling the decorations onto real-surfaces to obtain tangible reproductions.
△ Less
Submitted 1 February, 2021;
originally announced February 2021.
-
Shape retrieval of non-rigid 3d human models
Authors:
David Pickup,
Xianfang Sun,
Paul L Rosin,
Ralph R Martin,
Z Cheng,
Zhouhui Lian,
Masaki Aono,
A Ben Hamza,
A Bronstein,
M Bronstein,
S Bu,
Umberto Castellani,
S Cheng,
Valeria Garro,
Andrea Giachetti,
Afzal Godil,
Luca Isaia,
J Han,
Henry Johan,
L Lai,
Bo Li,
C Li,
Haisheng Li,
Roee Litman,
X Liu
, et al. (6 additional authors not shown)
Abstract:
3D models of humans are commonly used within computer graphics and vision, and so the ability to distinguish between body shapes is an important shape retrieval problem. We extend our recent paper which provided a benchmark for testing non-rigid 3D shape retrieval algorithms on 3D human models. This benchmark provided a far stricter challenge than previous shape benchmarks. We have added 145 new m…
▽ More
3D models of humans are commonly used within computer graphics and vision, and so the ability to distinguish between body shapes is an important shape retrieval problem. We extend our recent paper which provided a benchmark for testing non-rigid 3D shape retrieval algorithms on 3D human models. This benchmark provided a far stricter challenge than previous shape benchmarks. We have added 145 new models for use as a separate training set, in order to standardise the training data used and provide a fairer comparison. We have also included experiments with the FAUST dataset of human scans. All participants of the previous benchmark study have taken part in the new tests reported here, many providing updated results using the new data. In addition, further participants have also taken part, and we provide extra analysis of the retrieval results. A total of 25 different shape retrieval methods.
△ Less
Submitted 1 March, 2020;
originally announced March 2020.
-
Texel-Att: Representing and Classifying Element-based Textures by Attributes
Authors:
Marco Godi,
Christian Joppi,
Andrea Giachetti,
Fabio Pellacini,
Marco Cristani
Abstract:
Element-based textures are a kind of texture formed by nameable elements, the texels [1], distributed according to specific statistical distributions; it is of primary importance in many sectors, namely textile, fashion and interior design industry. State-of-theart texture descriptors fail to properly characterize element-based texture, so we present Texel-Att to fill this gap. Texel-Att is the fi…
▽ More
Element-based textures are a kind of texture formed by nameable elements, the texels [1], distributed according to specific statistical distributions; it is of primary importance in many sectors, namely textile, fashion and interior design industry. State-of-theart texture descriptors fail to properly characterize element-based texture, so we present Texel-Att to fill this gap. Texel-Att is the first fine-grained, attribute-based representation and classification framework for element-based textures. It first individuates texels, characterizing them with individual attributes; subsequently, texels are grouped and characterized through layout attributes, which give the Texel-Att representation. Texels are detected by a Mask-RCNN, trained on a brand-new element-based texture dataset, ElBa, containing 30K texture images with 3M fully-annotated texels. Examples of individual and layout attributes are exhibited to give a glimpse on the level of achievable graininess. In the experiments, we present detection results to show that texels can be precisely individuated, even on textures "in the wild"; to this sake, we individuate the element-based classes of the Describable Texture Dataset (DTD), where almost 900K texels have been manually annotated, leading to the Element-based DTD (E-DTD). Subsequently, classification and ranking results demonstrate the expressivity of Texel-Att on ElBa and E-DTD, overcoming the alternative features and relative attributes, doubling the best performance in some cases; finally, we report interactive search results on ElBa and E-DTD: with Texel-Att on the E-DTD dataset we are able to individuate within 10 iterations the desired texture in the 90% of cases, against the 71% obtained with a combination of the finest existing attributes so far. Dataset and code is available at https://github.com/godimarcovr/Texel-Att
△ Less
Submitted 30 August, 2019; v1 submitted 29 August, 2019;
originally announced August 2019.
-
Texture Retrieval in the Wild through detection-based attributes
Authors:
Christian Joppi,
Marco Godi,
Andrea Giachetti,
Fabio Pellacini,
Marco Cristani
Abstract:
Capturing the essence of a textile image in a robust way is important to retrieve it in a large repository, especially if it has been acquired in the wild (by taking a photo of the textile of interest). In this paper we show that a texel-based representation fits well with this task. In particular, we refer to Texel-Att, a recent texel-based descriptor which has shown to capture fine grained varia…
▽ More
Capturing the essence of a textile image in a robust way is important to retrieve it in a large repository, especially if it has been acquired in the wild (by taking a photo of the textile of interest). In this paper we show that a texel-based representation fits well with this task. In particular, we refer to Texel-Att, a recent texel-based descriptor which has shown to capture fine grained variations of a texture, for retrieval purposes. After a brief explanation of Texel-Att, we will show in our experiments that this descriptor is robust to distortions resulting from acquisitions in the wild by setting up an experiment in which textures from the ElBa (an Element-Based texture dataset) are artificially distorted and then used to retrieve the original image. We compare our approach with existing descriptors using a simple ranking framework based on distance functions. Results show that even under extreme conditions (such as a down-sampling with a factor of 10), we perform better than alternative approaches.
△ Less
Submitted 4 October, 2019; v1 submitted 29 August, 2019;
originally announced August 2019.
-
SIMCO: SIMilarity-based object COunting
Authors:
Marco Godi,
Christian Joppi,
Andrea Giachetti,
Marco Cristani
Abstract:
We present SIMCO, the first agnostic multi-class object counting approach. SIMCO starts by detecting foreground objects through a novel Mask RCNN-based architecture trained beforehand (just once) on a brand-new synthetic 2D shape dataset, InShape; the idea is to highlight every object resembling a primitive 2D shape (circle, square, rectangle, etc.). Each object detected is described by a low-dime…
▽ More
We present SIMCO, the first agnostic multi-class object counting approach. SIMCO starts by detecting foreground objects through a novel Mask RCNN-based architecture trained beforehand (just once) on a brand-new synthetic 2D shape dataset, InShape; the idea is to highlight every object resembling a primitive 2D shape (circle, square, rectangle, etc.). Each object detected is described by a low-dimensional embedding, obtained from a novel similarity-based head branch; this latter implements a triplet loss, encouraging similar objects (same 2D shape + color and scale) to map close. Subsequently, SIMCO uses this embedding for clustering, so that different types of objects can emerge and be counted, making SIMCO the very first multi-class unsupervised counter. Experiments show that SIMCO provides state-of-the-art scores on counting benchmarks and that it can also help in many challenging image understanding tasks.
△ Less
Submitted 13 October, 2020; v1 submitted 15 April, 2019;
originally announced April 2019.
