-
Understanding and Supporting Co-viewing Comedy in VR with Embodied Expressive Avatars
Authors:
Ryo Ohara,
Chi-Lan Yang,
Takuji Narumi,
Hideaki Kuzuoka
Abstract:
Co-viewing videos with family and friends remotely has become prevalent with the support of communication channels such as text messaging or real-time voice chat. However, current co-viewing platforms often lack visible embodied cues, such as body movements and facial expressions. This absence can reduce emotional engagement and the sense of co-presence when people are watching together remotely.…
▽ More
Co-viewing videos with family and friends remotely has become prevalent with the support of communication channels such as text messaging or real-time voice chat. However, current co-viewing platforms often lack visible embodied cues, such as body movements and facial expressions. This absence can reduce emotional engagement and the sense of co-presence when people are watching together remotely. Although virtual reality (VR) is an emerging technology that allows individuals to participate in various social activities while embodied as avatars, we still do not fully understand how this embodiment in VR affects co-viewing experiences, particularly in terms of engagement, emotional contagion, and expressive norms. In a controlled experiment involving eight triads of three participants each (N=24), we compared the participants' perceptions and reactions while watching comedy in VR using embodied expressive avatars that displayed visible laughter cues. This was contrasted with a control condition where no such embodied expressions were presented. With a mixed-method analysis, we found that embodied laughter cues shifted participants' engagement from individual immersion to socially coordinated participation. Participants reported heightened self-awareness of emotional expression, greater emotional contagion, and the development of expressive norms surrounding co-viewers' laughter. The result highlighted the tension between individual engagement and interpersonal emotional accommodation when co-viewing with embodied expressive avatars.
△ Less
Submitted 26 May, 2025;
originally announced May 2025.
-
LUIDA: Large-scale Unified Infrastructure for Digital Assessments based on Commercial Metaverse Platform
Authors:
Yong-Hao Hu,
Sotaro Yokoi,
Yuji Hatada,
Yuichi Hiroi,
Takuji Narumi,
Takefumi Hiraki
Abstract:
Online experiments using metaverse platforms have gained significant traction in Human-Computer Interaction and Virtual Reality (VR) research. However, current research workflows are highly fragmented, as researchers must use separate tools for system implementation, participant recruitment, experiment execution, and data collection, reducing consistency and increasing workload. We present LUIDA (…
▽ More
Online experiments using metaverse platforms have gained significant traction in Human-Computer Interaction and Virtual Reality (VR) research. However, current research workflows are highly fragmented, as researchers must use separate tools for system implementation, participant recruitment, experiment execution, and data collection, reducing consistency and increasing workload. We present LUIDA (Large-scale Unified Infrastructure for Digital Assessments), a metaverse-based framework that integrates these fragmented processes. LUIDA automatically allocates interconnected virtual environments for parallel experiment execution and provides implementation templates adaptable to various VR research domains, requiring minimal metaverse development expertise. Our evaluation included two studies using a prototype built on Cluster, the commercial metaverse platform. First, VR researchers using LUIDA to develop and run experiments reported high usability scores (SUS: 73.75) and moderate workload (NASA-TLX: 24.11) for overall usage, with interviews confirming streamlined workflows compared to traditional laboratory experiments. Second, we conducted three replicated experiments with public Cluster users, each recruiting approximately 200 participants within one week. These experiments produced results that closely matched the original studies, validating the experimental integrity of LUIDA across research domains. After technical refinements, we plan to release LUIDA as an open platform, providing a standardized protocol to improve research efficiency and experimental reproducibility in VR studies.
△ Less
Submitted 24 April, 2025;
originally announced April 2025.
-
Effects of Social Contextual Variation Using Partner Avatars on Memory Acquisition and Retention
Authors:
Takato Mizuho,
Takuji Narumi,
Hideaki Kuzuoka
Abstract:
This study investigates how partner avatar design affects learning and memory when an avatar serves as a lecturer. Based on earlier research on the environmental context dependency of memory, we hypothesize that the use of diverse partner avatars results in a slower learning rate but better memory retention than that of a constant partner avatar. Accordingly, participants were tasked with memorizi…
▽ More
This study investigates how partner avatar design affects learning and memory when an avatar serves as a lecturer. Based on earlier research on the environmental context dependency of memory, we hypothesize that the use of diverse partner avatars results in a slower learning rate but better memory retention than that of a constant partner avatar. Accordingly, participants were tasked with memorizing Tagalog--Japanese word pairs. On the first day of the experiment, they repeatedly learned the pairs over six sessions from a partner avatar in an immersive virtual environment. One week later, on the second day of the experiment, they underwent a recall test in a real environment. We employed a between-participants design to compare the following conditions: the varied avatar condition, in which each repetition used a different avatar, and the constant avatar condition, in which the same avatar was used throughout the experiment. Results showed that, compared to the constant avatar condition, the varied avatar condition resulted in significantly lower recall performance in the repeated learning trials conducted on the first day. However, the avatar conditions showed no significant differences in the final recall test on the second day. We discuss these effects in relation to the social presence of the partner avatar. This study opens up a novel approach to optimizing the effectiveness of instructor avatars in immersive virtual environments.
△ Less
Submitted 16 January, 2025;
originally announced January 2025.
-
Bridging Player Intentions: Exploring the Potential of Synchronized Haptic Controllers in Multiplayer Game
Authors:
Kenta Hashiura,
Kazuya Iida,
Takeru Hashimoto,
Youichi Kamiyama,
Keita Watanabe,
Kouta Minamizawa,
Takuji Narumi
Abstract:
In multiplayer cooperative video games, players traditionally use individual controllers, inferring others' actions through on-screen visuals and their own movements. This indirect understanding limits truly collaborative gameplay. Research in Joint Action shows that when manipulating a single object, motor performance improves when two people operate together while sensing each other's movements.…
▽ More
In multiplayer cooperative video games, players traditionally use individual controllers, inferring others' actions through on-screen visuals and their own movements. This indirect understanding limits truly collaborative gameplay. Research in Joint Action shows that when manipulating a single object, motor performance improves when two people operate together while sensing each other's movements. Building on this, we developed a controller allowing multiple players to operate simultaneously while sharing haptic sensations. We showcased our system at exhibitions, gathering feedback from over 150 participants on how shared sensory input affects their gaming experience. This approach could transform player interaction, enhance cooperation, and redefine multiplayer gaming experiences.
△ Less
Submitted 15 November, 2024; v1 submitted 7 November, 2024;
originally announced November 2024.
-
SealMates: Supporting Communication in Video Conferencing using a Collective Behavior-Driven Avatar
Authors:
Mark Armstrong,
Chi-Lan Yang,
Kinga Skiers,
Mengzhen Lim,
Tamil Selvan Gunasekaran,
Ziyue Wang,
Takuji Narumi,
Kouta Minamizawa,
Yun Suen Pai
Abstract:
The limited nonverbal cues and spatially distributed nature of remote communication make it challenging for unacquainted members to be expressive during social interactions over video conferencing. Though it enables seeing others' facial expressions, the visual feedback can instead lead to unexpected self-focus, resulting in users missing cues for others to engage in the conversation equally. To s…
▽ More
The limited nonverbal cues and spatially distributed nature of remote communication make it challenging for unacquainted members to be expressive during social interactions over video conferencing. Though it enables seeing others' facial expressions, the visual feedback can instead lead to unexpected self-focus, resulting in users missing cues for others to engage in the conversation equally. To support expressive communication and equal participation among unacquainted counterparts, we propose SealMates, a behavior-driven avatar in which the avatar infers the engagement level of the group based on collective gaze and speech patterns and then moves across interlocutors' windows in the video conferencing. By conducting a controlled experiment with 15 groups of triads, we found the avatar's movement encouraged people to experience more self-disclosure and made them perceive everyone was equally engaged in the conversation than when there was no behavior-driven avatar. We discuss how a behavior-driven avatar influences distributed members' perceptions and the implications of avatar-mediated communication for future platforms.
△ Less
Submitted 10 April, 2024;
originally announced April 2024.
-
Reduction of Forgetting by Contextual Variation During Encoding Using 360-Degree Video-Based Immersive Virtual Environments
Authors:
Takato Mizuho,
Takuji Narumi,
Hideaki Kuzuoka
Abstract:
Recall impairment in a different environmental context from learning is called context-dependent forgetting. Two learning methods have been proposed to prevent context-dependent forgetting: reinstatement and decontextualization. Reinstatement matches the environmental context between learning and retrieval, whereas decontextualization involves repeated learning in various environmental contexts an…
▽ More
Recall impairment in a different environmental context from learning is called context-dependent forgetting. Two learning methods have been proposed to prevent context-dependent forgetting: reinstatement and decontextualization. Reinstatement matches the environmental context between learning and retrieval, whereas decontextualization involves repeated learning in various environmental contexts and eliminates the context dependency of memory. Conventionally, these methods have been validated by switching between physical rooms. However, in this study, we use immersive virtual environments (IVEs) as the environmental context assisted by virtual reality (VR), which is known for its low cost and high reproducibility compared to traditional manipulation. Whereas most existing studies using VR have failed to reveal the reinstatement effect, we test its occurrence using a 360-degree video-based IVE with improved familiarity and realism instead of a computer graphics-based IVE. Furthermore, we are the first to address decontextualization using VR. Our experiment showed that repeated learning in the same constant IVE as retrieval did not significantly reduce forgetting compared to repeated learning in different constant IVEs. Conversely, repeated learning in various IVEs significantly reduced forgetting than repeated learning in constant IVEs. These findings contribute to the design of IVEs for VR-based applications, particularly in educational settings.
△ Less
Submitted 21 May, 2024; v1 submitted 7 April, 2024;
originally announced April 2024.
-
Perceptual Dimensions of Physical Properties of Handheld Objects Induced by Impedance Changes
Authors:
Takeru Hashimoto,
Shigeo Yoshida,
Takuji Narumi
Abstract:
Haptics in virtual reality is the emerging dimension after audiovisual experiences. Researchers designed several handheld VR controllers to simulate haptic experiences in virtual reality environments. Some of these devices, equipped to deliver active force, can dynamically alter the timing and intensity of force feedback, potentially offering a wide array of haptic sensations. Past research primar…
▽ More
Haptics in virtual reality is the emerging dimension after audiovisual experiences. Researchers designed several handheld VR controllers to simulate haptic experiences in virtual reality environments. Some of these devices, equipped to deliver active force, can dynamically alter the timing and intensity of force feedback, potentially offering a wide array of haptic sensations. Past research primarily used a single index to evaluate how users perceive physical property parameters, potentially limiting the assessment to the designer's intended scope and neglecting other potential perceptual experiences.
Therefore, this study evaluates not how much but how humans feel a physical property when stimuli are changed. We conducted interviews to investigate how people feel when a haptic device changes motion impedance. We used thematic analysis to abstract the results of the interviews and gain an understanding of how humans attribute force feedback to a phenomenon. We also generated a vocabulary from the themes obtained from the interviews and asked users to evaluate force feedback using the semantic difference method. A factor analysis was used to investigate how changing the basic elements of motion, such as inertia, viscosity, and stiffness of the motion system, affects haptic perception. As a result, we obtained four critical factors: size, viscosity, weight, and flexibility factor, and clarified the correspondence between these factors and the change of impedance.
△ Less
Submitted 20 March, 2025; v1 submitted 4 December, 2023;
originally announced December 2023.
-
Move or Push? Studying Pseudo-Haptic Perceptions Obtained with Motion or Force Input
Authors:
Yutaro Hirao,
Takuji Narumi,
Ferran Argelaguet,
Anatole Lecuyer
Abstract:
Pseudo-haptics techniques are interesting alternatives for generating haptic perceptions, which entails the manipulation of haptic perception through the appropriate alteration of primarily visual feedback in response to body movements. However, the use of pseudo-haptics techniques with a motion-input system can sometimes be limited. This paper investigates a novel approach for extending the poten…
▽ More
Pseudo-haptics techniques are interesting alternatives for generating haptic perceptions, which entails the manipulation of haptic perception through the appropriate alteration of primarily visual feedback in response to body movements. However, the use of pseudo-haptics techniques with a motion-input system can sometimes be limited. This paper investigates a novel approach for extending the potential of pseudo-haptics techniques in virtual reality (VR). The proposed approach utilizes a reaction force from force-input as a substitution of haptic cue for the pseudo-haptic perception. The paper introduced a manipulation method in which the vertical acceleration of the virtual hand is controlled by the extent of push-in of a force sensor. Such a force-input manipulation of a virtual body can not only present pseudo-haptics with less physical spaces and be used by more various users including physically handicapped people, but also can present the reaction force proportional to the user's input to the user. We hypothesized that such a haptic force cue would contribute to the pseudo-haptic perception. Therefore, the paper endeavors to investigate the force-input pseudo-haptic perception in a comparison with the motion-input pseudo-haptics. The paper compared force-input and motion-input manipulation in a point of achievable range and resolution of pseudo-haptic weight. The experimental results suggest that the force-input manipulation successfully extends the range of perceptible pseudo-weight by 80\% in comparison to the motion-input manipulation. On the other hand, it is revealed that the motion-input manipulation has 1 step larger number of distinguishable weight levels and is easier to operate than the force-input manipulation.
△ Less
Submitted 27 November, 2023;
originally announced November 2023.
-
Intranasal Chemosensory Lateralization Through the Multi-electrode Transcutaneous Electrical Nasal Bridge Stimulation
Authors:
Ayari Matsui,
Tomohiro Amemiya,
Takuji Narumi,
Kazuma Aoyama
Abstract:
Numerous studies have been conducted on display techniques for intranasal chemosensory perception. However, a limited number of studies have focused on the presentation of sensory spatial information. To artificially produce intranasal chemosensory spatial perception, we focused on a technique to induce intranasal chemosensation by transcutaneous electrical stimulation between the nasal bridge and…
▽ More
Numerous studies have been conducted on display techniques for intranasal chemosensory perception. However, a limited number of studies have focused on the presentation of sensory spatial information. To artificially produce intranasal chemosensory spatial perception, we focused on a technique to induce intranasal chemosensation by transcutaneous electrical stimulation between the nasal bridge and the back of the neck. Whether this technique stimulates the trigeminal nerve or the olfactory nerve remains debatable; if this method stimulates the trigeminal nerve, the differences in the amount of stimulation to the left and right trigeminal branches would evoke lateralization of intranasal chemosensory perception. Therefore, we propose a novel method to lateralize intranasal chemosensation by selectively stimulating the left or right trigeminal nerve branches through the shifting of an electrode on the nasal bridge to the left or right. Finite element simulations reveal that electrical stimulation applied between the electrodes on the left/right nasal bridge and the back of the neck results in the construction of a high current density area on the left/right branch of the trigeminal nerve. The results of two psychophysical experiments reveal that intranasal chemosensation can be lateralized by using the proposed method. The results of our experiment also suggest that lateralization is not the result of electrically induced tactile sensation of the skin surface but rather due to the distribution of stimuli to the trigeminal nerves. To the best of our knowledge, this study is the first successful lateralization of intranasal chemosensation that utilizes an easy-to-apply method without involving nostril blocking.
△ Less
Submitted 4 September, 2023;
originally announced September 2023.
-
Leveraging Tendon Vibration to Enhance Pseudo-Haptic Perceptions in VR
Authors:
Yutaro Hirao,
Tomohiro Amemiya,
Takuji Narumi,
Ferran Argelaguet,
Anatole Lécuyer
Abstract:
Pseudo-haptic techniques are used to modify haptic perception by appropriately changing visual feedback to body movements. Based on the knowledge that tendon vibration can affect our somatosensory perception, this paper proposes a method for leveraging tendon vibration to enhance pseudo-haptics during free arm motion. Three experiments were performed to examine the impact of tendon vibration on th…
▽ More
Pseudo-haptic techniques are used to modify haptic perception by appropriately changing visual feedback to body movements. Based on the knowledge that tendon vibration can affect our somatosensory perception, this paper proposes a method for leveraging tendon vibration to enhance pseudo-haptics during free arm motion. Three experiments were performed to examine the impact of tendon vibration on the range and resolution of pseudo-haptics. The first experiment investigated the effect of tendon vibration on the detection threshold of the discrepancy between visual and physical motion. The results indicated that vibrations applied to the inner tendons of the wrist and elbow increased the threshold, suggesting that tendon vibration can augment the applicable visual motion gain by approximately 13\% without users detecting the visual/physical discrepancy. Furthermore, the results demonstrate that tendon vibration acts as noise on haptic motion cues. The second experiment assessed the impact of tendon vibration on the resolution of pseudo-haptics by determining the just noticeable difference in pseudo-weight perception. The results suggested that the tendon vibration does not largely compromise the resolution of pseudo-haptics. The third experiment evaluated the equivalence between the weight perception triggered by tendon vibration and that by visual motion gain, that is, the point of subjective equality. The results revealed that vibration amplifies the weight perception and its effect was equivalent to that obtained using a gain of 0.64 without vibration, implying that the tendon vibration also functions as an additional haptic cue. Our results provide design guidelines and future work for enhancing pseudo-haptics with tendon vibration.
△ Less
Submitted 29 August, 2023; v1 submitted 1 September, 2022;
originally announced September 2022.
-
Revisiting Walking-in-Place by Introducing Step-Height Control, Elastic Input, and Pseudo-Haptic Feedback
Authors:
Yutaro Hirao,
Takuji Narumi,
Ferran Argelaguet,
Anatole Lécuyer
Abstract:
Walking-in-place (WIP) is a locomotion technique that enables users to "walk infinitely" through vast virtual environments using walking-like gestures within a limited physical space. This paper investigates alternative interaction schemes for WIP, addressing successively the control, input, and output of WIP. First, we introduce a novel height-based control to increase advanced speed. Second, we…
▽ More
Walking-in-place (WIP) is a locomotion technique that enables users to "walk infinitely" through vast virtual environments using walking-like gestures within a limited physical space. This paper investigates alternative interaction schemes for WIP, addressing successively the control, input, and output of WIP. First, we introduce a novel height-based control to increase advanced speed. Second, we introduce a novel input system for WIP based on elastic and passive strips. Third, we introduce the use of pseudo-haptic feedback as a novel output for WIP meant to alter walking sensations. The results of a series of user studies show that height and frequency based control of WIP can facilitate higher virtual speed with greater efficacy and ease than in frequency-based WIP. Second, using an upward elastic input system can result in a stable virtual speed control, although excessively strong elastic forces may impact the usability and user experience. Finally, using a pseudo-haptic approach can improve the perceived realism of virtual slopes. Taken together, our results suggest that, for future VR applications, there is value in further research into the use of alternative interaction schemes for walking-in-place.
△ Less
Submitted 29 August, 2023; v1 submitted 10 May, 2022;
originally announced May 2022.
-
Visual-Tactile Cross-Modal Data Generation using Residue-Fusion GAN with Feature-Matching and Perceptual Losses
Authors:
Shaoyu Cai,
Kening Zhu,
Yuki Ban,
Takuji Narumi
Abstract:
Existing psychophysical studies have revealed that the cross-modal visual-tactile perception is common for humans performing daily activities. However, it is still challenging to build the algorithmic mapping from one modality space to another, namely the cross-modal visual-tactile data translation/generation, which could be potentially important for robotic operation. In this paper, we propose a…
▽ More
Existing psychophysical studies have revealed that the cross-modal visual-tactile perception is common for humans performing daily activities. However, it is still challenging to build the algorithmic mapping from one modality space to another, namely the cross-modal visual-tactile data translation/generation, which could be potentially important for robotic operation. In this paper, we propose a deep-learning-based approach for cross-modal visual-tactile data generation by leveraging the framework of the generative adversarial networks (GANs). Our approach takes the visual image of a material surface as the visual data, and the accelerometer signal induced by the pen-sliding movement on the surface as the tactile data. We adopt the conditional-GAN (cGAN) structure together with the residue-fusion (RF) module, and train the model with the additional feature-matching (FM) and perceptual losses to achieve the cross-modal data generation. The experimental results show that the inclusion of the RF module, and the FM and the perceptual losses significantly improves cross-modal data generation performance in terms of the classification accuracy upon the generated data and the visual similarity between the ground-truth and the generated data.
△ Less
Submitted 12 July, 2021;
originally announced July 2021.
-
Virtual Co-Embodiment: Evaluation of the Sense of Agency while Sharing the Control of a Virtual Body among Two Individuals
Authors:
Rebecca Fribourg,
Nami Ogawa,
Ludovic Hoyet,
Ferran Argelaguet,
Takuji Narumi,
Michitaka Hirose,
Anatole Lécuyer
Abstract:
In this paper, we introduce a concept called ''virtual co-embodiment'', which enables a user to share their virtual avatar with another entity (e.g., another user, robot, or autonomous agent). We describe a proof-of-concept in which two users can be immersed from a first-person perspective in a virtual environment and can have complementary levels of control (total, partial, or none) over a shared…
▽ More
In this paper, we introduce a concept called ''virtual co-embodiment'', which enables a user to share their virtual avatar with another entity (e.g., another user, robot, or autonomous agent). We describe a proof-of-concept in which two users can be immersed from a first-person perspective in a virtual environment and can have complementary levels of control (total, partial, or none) over a shared avatar. In addition, we conducted an experiment to investigate the influence of users' level of control over the shared avatar and prior knowledge of their actions on the users' sense of agency and motor actions. The results showed that participants are good at estimating their real level of control but significantly overestimate their sense of agency when they can anticipate the motion of the avatar. Moreover, participants performed similar body motions regardless of their real control over the avatar. The results also revealed that the internal dimension of the locus of control, which is a personality trait, is negatively correlated with the user's perceived level of control. The combined results unfold a new range of applications in the fields of virtual-reality-based training and collaborative teleoperation, where users would be able to share their virtual body.
△ Less
Submitted 9 June, 2020; v1 submitted 8 November, 2019;
originally announced November 2019.
-
Redirection Controller Using Reinforcement Learning
Authors:
Yuchen Chang,
Keigo Matsumoto,
Takuji Narumi,
Tomohiro Tanikawa,
Michitaka Hirose
Abstract:
There is a growing demand for redirected walking (RDW) techniques and their application. To apply appropriate RDW methods and manipulation, the RDW controllers are predominantly used. There are three types of RDW controllers: direct scripted controller, generalized controller, and predictive controller. The scripted controller type pre-scripts the mapping between the real and virtual environments.…
▽ More
There is a growing demand for redirected walking (RDW) techniques and their application. To apply appropriate RDW methods and manipulation, the RDW controllers are predominantly used. There are three types of RDW controllers: direct scripted controller, generalized controller, and predictive controller. The scripted controller type pre-scripts the mapping between the real and virtual environments. The generalized controller type employs the RDW method and manipulation quantities according to a certain procedure depending on the user's position in relation to the real space. This approach has the potential to be reused in any environment; however, it is not fully optimized. The predictive controller type predicts the user's future path using the user's behavior and manages RDW techniques. This approach is highly anticipated to be very effective and versatile; however, it has not been sufficiently developed. This paper proposes a novel RDW controller using reinforcement learning (RL) with advanced plannability/versatility. Our simulation experiments indicate that the proposed method can reduce the number of reset manipulations, which is one of the indicators of the effectiveness of the RDW controller, compared to the generalized controller under real environments with many obstacles. Meanwhile, the experimental results also showed that the gain output by the proposed method oscillates. The results of a user study conducted showed that the proposed RDW controller can reduce the number of resets compared to the conventional generalized controller. Furthermore, no adverse effects such as cybersickness associated with the oscillation of the output gain were evinced. The simulation and user studies demonstrate that the proposed RDW controller with RL outperforms the existing generalized controllers and can be applied to users.
△ Less
Submitted 12 October, 2021; v1 submitted 20 September, 2019;
originally announced September 2019.
