-
Enhancing DMI Interactions by Integrating Haptic Feedback for Intricate Vibrato Technique
Authors:
Ziyue Piao,
Christian Frisson,
Bavo Van Kerrebroeck,
Marcelo M. Wanderley
Abstract:
This paper investigates the integration of force feedback in Digital Musical Instruments (DMI), specifically evaluating the reproduction of intricate vibrato techniques using haptic feedback controllers. We introduce our system for vibrato modulation using force feedback, composed of Bend-aid (a web-based sequencer platform using pre-designed haptic feedback models) and TorqueTuner (an open-source…
▽ More
This paper investigates the integration of force feedback in Digital Musical Instruments (DMI), specifically evaluating the reproduction of intricate vibrato techniques using haptic feedback controllers. We introduce our system for vibrato modulation using force feedback, composed of Bend-aid (a web-based sequencer platform using pre-designed haptic feedback models) and TorqueTuner (an open-source 1 Degree-of-Freedom (DoF) rotary haptic device for generating programmable haptic effects). We designed a formal user study to assess the impact of each haptic mode on user experience in a vibrato mimicry task. Twenty musically trained participants rated their user experience for the three haptic modes (Smooth, Detent, and Spring) using four Likert-scale scores: comfort, flexibility, ease of control, and helpfulness for the task. Finally, we asked participants to share their reflections. Our research indicates that while the Spring mode can help with light vibrato, preferences for haptic modes vary based on musical training background. This emphasizes the need for adaptable task interfaces and flexible haptic feedback in DMI design.
△ Less
Submitted 16 May, 2024;
originally announced May 2024.
-
The virtual drum circle: polyrhythmic music interactions in extended reality
Authors:
Bavo Van Kerrebroeck,
Kristel Crombé,
Stéphanie Wilain,
Marc Leman,
Pieter-Jan Maes
Abstract:
Emerging technologies in the domain of extended reality offer rich, new possibilities for the study and practice of joint music performance. Apart from the technological challenges, bringing music players together in extended reality raises important questions on their performance and embodied coordination. In this study, we designed an extended reality platform to assess a remote, bidirectional p…
▽ More
Emerging technologies in the domain of extended reality offer rich, new possibilities for the study and practice of joint music performance. Apart from the technological challenges, bringing music players together in extended reality raises important questions on their performance and embodied coordination. In this study, we designed an extended reality platform to assess a remote, bidirectional polyrhythmic interaction between two players, mediated in real time by their three-dimensional embodied avatars and a shared, virtual drum circle. We leveraged a multi-layered analysis framework to assess their performance quality, embodied co-regulation and first-person interaction experience, using statistical techniques for time-series analysis and mixed-effect regression and focusing on contrasts of visual coupling (not seeing / seeing as avatars / seeing as real) and auditory context (metronome / music). Results reveal that an auditory context with music improved the performance output as measured by a prediction error, increased movement energy and levels of experienced agency. Visual coupling impacted experiential qualities and induced prosocial effects with increased levels of partner realism resulting in increased levels of shared agency and self-other merging. Embodied co-regulation between players was impacted by auditory context and visual coupling, suggesting prediction-based compensatory mechanisms to deal with the novelty, difficulty, and expressivity in the musical interaction. This study contributes to the understanding of music performance in extended reality by using a methodological approach to demonstrate how co-regulation between players is impacted by visual coupling and auditory context and provides a basis and future directions for further action-oriented research.
△ Less
Submitted 30 August, 2023; v1 submitted 3 August, 2023;
originally announced August 2023.
-
Designing Deep Reinforcement Learning for Human Parameter Exploration
Authors:
Hugo Scurto,
Bavo Van Kerrebroeck,
Baptiste Caramiaux,
Frédéric Bevilacqua
Abstract:
Software tools for generating digital sound often present users with high-dimensional, parametric interfaces, that may not facilitate exploration of diverse sound designs. In this paper, we propose to investigate artificial agents using deep reinforcement learning to explore parameter spaces in partnership with users for sound design. We describe a series of user-centred studies to probe the creat…
▽ More
Software tools for generating digital sound often present users with high-dimensional, parametric interfaces, that may not facilitate exploration of diverse sound designs. In this paper, we propose to investigate artificial agents using deep reinforcement learning to explore parameter spaces in partnership with users for sound design. We describe a series of user-centred studies to probe the creative benefits of these agents and adapting their design to exploration. Preliminary studies observing users' exploration strategies with parametric interfaces and testing different agent exploration behaviours led to the design of a fully-functioning prototype, called Co-Explorer, that we evaluated in a workshop with professional sound designers. We found that the Co-Explorer enables a novel creative workflow centred on human-machine partnership, which has been positively received by practitioners. We also highlight varied user exploration behaviors throughout partnering with our system. Finally, we frame design guidelines for enabling such co-exploration workflow in creative digital applications.
△ Less
Submitted 28 January, 2021; v1 submitted 1 July, 2019;
originally announced July 2019.
