-
Exploring VQ-VAE with Prosody Parameters for Speaker Anonymization
Authors:
Sotheara Leang,
Anderson Augusma,
Eric Castelli,
Frédérique Letué,
Sethserey Sam,
Dominique Vaufreydaz
Abstract:
Human speech conveys prosody, linguistic content, and speaker identity. This article investigates a novel speaker anonymization approach using an end-to-end network based on a Vector-Quantized Variational Auto-Encoder (VQ-VAE) to deal with these speech components. This approach is designed to disentangle these components to specifically target and modify the speaker identity while preserving the l…
▽ More
Human speech conveys prosody, linguistic content, and speaker identity. This article investigates a novel speaker anonymization approach using an end-to-end network based on a Vector-Quantized Variational Auto-Encoder (VQ-VAE) to deal with these speech components. This approach is designed to disentangle these components to specifically target and modify the speaker identity while preserving the linguistic and emotionalcontent. To do so, three separate branches compute embeddings for content, prosody, and speaker identity respectively. During synthesis, taking these embeddings, the decoder of the proposed architecture is conditioned on both speaker and prosody information, allowing for capturing more nuanced emotional states and precise adjustments to speaker identification. Findings indicate that this method outperforms most baseline techniques in preserving emotional information. However, it exhibits more limited performance on other voice privacy tasks, emphasizing the need for further improvements.
△ Less
Submitted 24 September, 2024;
originally announced September 2024.
-
Wild Berry image dataset collected in Finnish forests and peatlands using drones
Authors:
Luigi Riz,
Sergio Povoli,
Andrea Caraffa,
Davide Boscaini,
Mohamed Lamine Mekhalfi,
Paul Chippendale,
Marjut Turtiainen,
Birgitta Partanen,
Laura Smith Ballester,
Francisco Blanes Noguera,
Alessio Franchi,
Elisa Castelli,
Giacomo Piccinini,
Luca Marchesotti,
Micael Santos Couceiro,
Fabio Poiesi
Abstract:
Berry picking has long-standing traditions in Finland, yet it is challenging and can potentially be dangerous. The integration of drones equipped with advanced imaging techniques represents a transformative leap forward, optimising harvests and promising sustainable practices. We propose WildBe, the first image dataset of wild berries captured in peatlands and under the canopy of Finnish forests u…
▽ More
Berry picking has long-standing traditions in Finland, yet it is challenging and can potentially be dangerous. The integration of drones equipped with advanced imaging techniques represents a transformative leap forward, optimising harvests and promising sustainable practices. We propose WildBe, the first image dataset of wild berries captured in peatlands and under the canopy of Finnish forests using drones. Unlike previous and related datasets, WildBe includes new varieties of berries, such as bilberries, cloudberries, lingonberries, and crowberries, captured under severe light variations and in cluttered environments. WildBe features 3,516 images, including a total of 18,468 annotated bounding boxes. We carry out a comprehensive analysis of WildBe using six popular object detectors, assessing their effectiveness in berry detection across different forest regions and camera types. WildBe is publicly available on HuggingFace at https://huggingface.co/datasets/FBK-TeV/WildBe.
△ Less
Submitted 24 February, 2025; v1 submitted 13 May, 2024;
originally announced May 2024.
-
Preliminary Study on SSCF-derived Polar Coordinate for ASR
Authors:
Sotheara Leang,
Eric Castelli,
Dominique Vaufreydaz,
Sethserey Sam
Abstract:
The transition angles are defined to describe the vowel-to-vowel transitions in the acoustic space of the Spectral Subband Centroids, and the findings show that they are similar among speakers and speaking rates. In this paper, we propose to investigate the usage of polar coordinates in favor of angles to describe a speech signal by characterizing its acoustic trajectory and using them in Automati…
▽ More
The transition angles are defined to describe the vowel-to-vowel transitions in the acoustic space of the Spectral Subband Centroids, and the findings show that they are similar among speakers and speaking rates. In this paper, we propose to investigate the usage of polar coordinates in favor of angles to describe a speech signal by characterizing its acoustic trajectory and using them in Automatic Speech Recognition. According to the experimental results evaluated on the BRAF100 dataset, the polar coordinates achieved significantly higher accuracy than the angles in the mixed and cross-gender speech recognitions, demonstrating that these representations are superior at defining the acoustic trajectory of the speech signal. Furthermore, the accuracy was significantly improved when they were utilized with their first and second-order derivatives ($Δ$, $Δ$$Δ$), especially in cross-female recognition. However, the results showed they were not much more gender-independent than the conventional Mel-frequency Cepstral Coefficients (MFCCs).
△ Less
Submitted 30 November, 2022;
originally announced December 2022.
-
Smartphone-based user positioning in a multiple-user context with Wi-Fi and Bluetooth
Authors:
Viet-Cuong Ta,
Trung-Kien Dao,
Dominique Vaufreydaz,
Eric Castelli
Abstract:
In a multiuser context, the Bluetooth data from the smartphone could give an approximation of the distance between users. Meanwhile, the Wi-Fi data can be used to calculate the user's position directly. However, both the Wi-Fi-based position outputs and Bluetooth-based distances are affected by some degree of noise. In our work, we propose several approaches to combine the two types of outputs for…
▽ More
In a multiuser context, the Bluetooth data from the smartphone could give an approximation of the distance between users. Meanwhile, the Wi-Fi data can be used to calculate the user's position directly. However, both the Wi-Fi-based position outputs and Bluetooth-based distances are affected by some degree of noise. In our work, we propose several approaches to combine the two types of outputs for improving the tracking accuracy in the context of collaborative positioning. The two proposed approaches attempt to build a model for measuring the errors of the Bluetooth output and Wi-Fi output. In a non-temporal approach, the model establishes the relationship in a specific interval of the Bluetooth output and Wi-Fi output. In a temporal approach, the error measurement model is expanded to include the time component between users' movement. To evaluate the performance of the two approaches, we collected the data from several multiuser scenarios in indoor environment. The results show that the proposed approaches could reach a distance error around 3.0m for 75 percent of time, which outperforms the positioning results of the standard Wi-Fi fingerprinting model.
△ Less
Submitted 16 July, 2018;
originally announced July 2018.
