Audio and Speech Processing

Authors and titles for January 2024

[151] arXiv:2401.04210 (cross-list from cs.CV) [pdf, html, other]

Title: FunnyNet-W: Multimodal Learning of Funny Moments in Videos in the Wild

Zhi-Song Liu, Robin Courant, Vicky Kalogeiton

Comments: 22 pages, 14 figures

Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Multimedia (cs.MM); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[152] arXiv:2401.04235 (cross-list from cs.CL) [pdf, html, other]

Title: High-precision Voice Search Query Correction via Retrievable Speech-text Embedings

Christopher Li, Gary Wang, Kyle Kastner, Heng Su, Allen Chen, Andrew Rosenberg, Zhehuai Chen, Zelin Wu, Leonid Velikovich, Pat Rondon, Diamantino Caseiro, Petar Aleksic

Subjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[153] arXiv:2401.04389 (cross-list from cs.SD) [pdf, html, other]

Title: RaD-Net: A Repairing and Denoising Network for Speech Signal Improvement

Mingshuai Liu, Zhuangqi Chen, Xiaopeng Yan, Yuanjun Lv, Xianjun Xia, Chuanzeng Huang, Yijian Xiao, Lei Xie

Comments: submitted to ICASSP 2024

Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)

[154] arXiv:2401.04394 (cross-list from cs.MM) [pdf, html, other]

Title: SonicVisionLM: Playing Sound with Vision Language Models

Zhifeng Xie, Shengye Yu, Qile He, Mengtian Li

Comments: CVPR 2024

Subjects: Multimedia (cs.MM); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[155] arXiv:2401.04558 (cross-list from cs.SD) [pdf, html, other]

Title: HyperGANStrument: Instrument Sound Synthesis and Editing with Pitch-Invariant Hypernetworks

Zhe Zhang, Taketo Akama

Comments: 5 pages, 3 figures, Accepted for 2024 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Audio examples: this https URL

Subjects: Sound (cs.SD); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

[156] arXiv:2401.04577 (cross-list from cs.SD) [pdf, html, other]

Title: Masked Audio Generation using a Single Non-Autoregressive Transformer

Alon Ziv, Itai Gat, Gael Le Lan, Tal Remez, Felix Kreuk, Alexandre Défossez, Jade Copet, Gabriel Synnaeve, Yossi Adi

Subjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

[157] arXiv:2401.04737 (cross-list from cs.SD) [pdf, other]

Title: Music Genre Classification: A Comparative Analysis of CNN and XGBoost Approaches with Mel-frequency cepstral coefficients and Mel Spectrograms

Yigang Meng

Subjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Audio and Speech Processing (eess.AS)

[158] arXiv:2401.04747 (cross-list from cs.SD) [pdf, html, other]

Title: DiffSHEG: A Diffusion-Based Approach for Real-Time Speech-driven Holistic 3D Expression and Gesture Generation

Junming Chen, Yunfei Liu, Jianan Wang, Ailing Zeng, Yu Li, Qifeng Chen

Comments: Accepted by CVPR 2024. Project page: this https URL

Subjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Graphics (cs.GR); Audio and Speech Processing (eess.AS)

[159] arXiv:2401.04868 (cross-list from cs.CL) [pdf, html, other]

Title: Real-time and Continuous Turn-taking Prediction Using Voice Activity Projection

Koji Inoue, Bing'er Jiang, Erik Ekstedt, Tatsuya Kawahara, Gabriel Skantze

Comments: This paper has been accepted for presentation at International Workshop on Spoken Dialogue Systems Technology 2024 (IWSDS 2024) and represents the author's version of the work

Subjects: Computation and Language (cs.CL); Human-Computer Interaction (cs.HC); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[160] arXiv:2401.04935 (cross-list from cs.MM) [pdf, html, other]

Title: Learning Audio Concepts from Counterfactual Natural Language

Ali Vosoughi, Luca Bondi, Ho-Hsiang Wu, Chenliang Xu

Comments: Accepted at ICASSP 2024

Subjects: Multimedia (cs.MM); Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[161] arXiv:2401.04964 (cross-list from eess.SP) [pdf, other]

Title: Self-supervised speech representation and contextual text embedding for match-mismatch classification with EEG recording

Bo Wang, Xiran Xu, Zechen Zhang, Haolin Zhu, YuJie Yan, Xihong Wu, Jing Chen

Comments: 2 pages, 2 figures, accepted by ICASSP 2024

Subjects: Signal Processing (eess.SP); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[162] arXiv:2401.05060 (cross-list from cs.SD) [pdf, html, other]

Title: MuTox: Universal MUltilingual Audio-based TOXicity Dataset and Zero-shot Detector

Marta R. Costa-jussà, Mariano Coria Meglioli, Pierre Andrews, David Dale, Prangthip Hansanti, Elahe Kalbassi, Alex Mourachko, Christophe Ropers, Carleigh Wood

Subjects: Sound (cs.SD); Computation and Language (cs.CL); Audio and Speech Processing (eess.AS)

[163] arXiv:2401.05064 (cross-list from cs.SD) [pdf, html, other]

Title: Singer Identity Representation Learning using Self-Supervised Techniques

Bernardo Torres, Stefan Lattner, Gaël Richard

Comments: Accepted at the ISMIR conference, Milan, Italy, 2023

Journal-ref: Proceedings of the 24th International Society for Music Information Retrieval Conference (ISMIR 2023), Milan, Italy

Subjects: Sound (cs.SD); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

[164] arXiv:2401.05111 (cross-list from cs.SD) [pdf, html, other]

Title: Noise-robust zero-shot text-to-speech synthesis conditioned on self-supervised speech-representation model with adapters

Kenichi Fujita, Hiroshi Sato, Takanori Ashihara, Hiroki Kanagawa, Marc Delcroix, Takafumi Moriya, Yusuke Ijima

Comments: 5 pages,3 figures, Accepted to IEEE ICASSP 2024

Subjects: Sound (cs.SD); Computation and Language (cs.CL); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

[165] arXiv:2401.05531 (cross-list from cs.LG) [pdf, html, other]

Title: VI-PANN: Harnessing Transfer Learning and Uncertainty-Aware Variational Inference for Improved Generalization in Audio Pattern Recognition

John Fischer, Marko Orescanin, Eric Eckstrand

Comments: Published in IEEE Access

Journal-ref: IEEE Access (2024)

Subjects: Machine Learning (cs.LG); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[166] arXiv:2401.05551 (cross-list from cs.CL) [pdf, html, other]

Title: Useful Blunders: Can Automated Speech Recognition Errors Improve Downstream Dementia Classification?

Changye Li, Weizhe Xu, Trevor Cohen, Serguei Pakhomov

Comments: To appear on Journal of Biomedical Informatics

Subjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[167] arXiv:2401.05614 (cross-list from cs.SD) [pdf, other]

Title: Self-Attention and Hybrid Features for Replay and Deep-Fake Audio Detection

Lian Huang, Chi-Man Pun

Subjects: Sound (cs.SD); Multimedia (cs.MM); Audio and Speech Processing (eess.AS)

[168] arXiv:2401.05689 (cross-list from cs.CL) [pdf, html, other]

Title: UCorrect: An Unsupervised Framework for Automatic Speech Recognition Error Correction

Jiaxin Guo, Minghan Wang, Xiaosong Qiao, Daimeng Wei, Hengchao Shang, Zongyao Li, Zhengzhe Yu, Yinglu Li, Chang Su, Min Zhang, Shimin Tao, Hao Yang

Comments: Accepted in ICASSP 2023

Subjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[169] arXiv:2401.05698 (cross-list from cs.CV) [pdf, html, other]

Title: HiCMAE: Hierarchical Contrastive Masked Autoencoder for Self-Supervised Audio-Visual Emotion Recognition

Licai Sun, Zheng Lian, Bin Liu, Jianhua Tao

Comments: Accepted by Information Fusion. The code is available at this https URL

Journal-ref: Information Fusion, 2024

Subjects: Computer Vision and Pattern Recognition (cs.CV); Human-Computer Interaction (cs.HC); Multimedia (cs.MM); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[170] arXiv:2401.05757 (cross-list from cs.SD) [pdf, other]

Title: Intuitive Control of Scraping and Rubbing Through Audio-tactile Synthesis

Mitsuko Aramaki (PRISM), Corentin Bernard (PRISM), Richard Kronland-Martinet (PRISM), Samuel Poirot (PRISM), Sølvi Ystad (PRISM)

Journal-ref: 16th International Symposium on CMMR, Future University Hakodate, Japan; Asia University, Japan; Nihon University, Japan; Laboratoire PRISM, Marseille, France, Nov 2023, Tokyo, France

Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS); Classical Physics (physics.class-ph)

[171] arXiv:2401.05850 (cross-list from cs.SD) [pdf, html, other]

Title: Contrastive Loss Based Frame-wise Feature disentanglement for Polyphonic Sound Event Detection

Yadong Guan, Jiqing Han, Hongwei Song, Wenjie Song, Guibin Zheng, Tieran Zheng, Yongjun He

Comments: accepted by icassp2024

Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)

[172] arXiv:2401.06390 (cross-list from cs.SD) [pdf, html, other]

Title: LCB-net: Long-Context Biasing for Audio-Visual Speech Recognition

Fan Yu, Haoxu Wang, Xian Shi, Shiliang Zhang

Comments: Accepted by ICASPP 2024

Subjects: Sound (cs.SD); Multimedia (cs.MM); Audio and Speech Processing (eess.AS)

[173] arXiv:2401.06832 (cross-list from cs.CL) [pdf, other]

Title: XLS-R Deep Learning Model for Multilingual ASR on Low- Resource Languages: Indonesian, Javanese, and Sundanese

Panji Arisaputra, Alif Tri Handoyo, Amalia Zahra

Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[174] arXiv:2401.06913 (cross-list from cs.SD) [pdf, html, other]

Title: Microphone Conversion: Mitigating Device Variability in Sound Event Classification

Myeonghoon Ryu, Hongseok Oh, Suji Lee, Han Park

Comments: Accepted to ICASSP 2024

Subjects: Sound (cs.SD); Machine Learning (cs.LG); Multimedia (cs.MM); Audio and Speech Processing (eess.AS)

[175] arXiv:2401.07084 (cross-list from cs.MM) [pdf, html, other]

Title: ScripTONES: Sentiment-Conditioned Music Generation for Movie Scripts

Vishruth Veerendranath, Vibha Masti, Utkarsh Gupta, Hrishit Chaudhuri, Gowri Srinivasa

Comments: Presented at NeurIPS 2023 - ML For Audio workshop. To appear in proceedings of AIML Systems 2023 - Generative AI

Subjects: Multimedia (cs.MM); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[176] arXiv:2401.07333 (cross-list from cs.CL) [pdf, html, other]

Title: ELLA-V: Stable Neural Codec Language Modeling with Alignment-guided Sequence Reordering

Yakun Song, Zhuo Chen, Xiaofei Wang, Ziyang Ma, Xie Chen

Comments: Working in progress

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[177] arXiv:2401.07360 (cross-list from cs.CL) [pdf, html, other]

Title: Promptformer: Prompted Conformer Transducer for ASR

Sergio Duarte-Torres, Arunasish Sen, Aman Rana, Lukas Drude, Alejandro Gomez-Alanis, Andreas Schwarz, Leif Rädel, Volker Leutnant

Subjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[178] arXiv:2401.07532 (cross-list from cs.SD) [pdf, html, other]

Title: Multi-view MidiVAE: Fusing Track- and Bar-view Representations for Long Multi-track Symbolic Music Generation

Zhiwei Lin, Jun Chen, Boshi Tang, Binzhu Sha, Jing Yang, Yaolong Ju, Fan Fan, Shiyin Kang, Zhiyong Wu, Helen Meng

Comments: Accepted by ICASSP 2024

Subjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Audio and Speech Processing (eess.AS)

[179] arXiv:2401.07575 (cross-list from cs.CL) [pdf, html, other]

Title: Cascaded Cross-Modal Transformer for Audio-Textual Classification

Nicolae-Catalin Ristea, Andrei Anghel, Radu Tudor Ionescu

Comments: Accepted for publication in Artificial Intelligence Review

Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[180] arXiv:2401.07879 (cross-list from cs.SD) [pdf, other]

Title: Decoupled Spatial and Temporal Processing for Resource Efficient Multichannel Speech Enhancement

Ashutosh Pandey, Buye Xu

Comments: Accepted for publication at ICASSP

Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)

[181] arXiv:2401.07882 (cross-list from cs.SD) [pdf, other]

Title: On the Importance of Neural Wiener Filter for Resource Efficient Multichannel Speech Enhancement

Tsun-An Hsieh, Jacob Donley, Daniel Wong, Buye Xu, Ashutosh Pandey

Comments: Accepted for publication at ICASSP

Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)

[182] arXiv:2401.07957 (cross-list from eess.IV) [pdf, other]

Title: Machine Perceptual Quality: Evaluating the Impact of Severe Lossy Compression on Audio and Image Models

Dan Jacobellis, Daniel Cummings, Neeraja J. Yadwadkar

Comments: 10 pages; abridged version published in IEEE Data Compression Conference 2024

Subjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[183] arXiv:2401.07967 (cross-list from cs.SD) [pdf, other]

Title: MCMChaos: Improvising Rap Music with MCMC Methods and Chaos Theory

Robert G. Kimelman

Subjects: Sound (cs.SD); Computation and Language (cs.CL); Human-Computer Interaction (cs.HC); Audio and Speech Processing (eess.AS)

[184] arXiv:2401.08049 (cross-list from cs.CV) [pdf, other]

Title: EmoTalker: Emotionally Editable Talking Face Generation via Diffusion Model

Bingyuan Zhang, Xulong Zhang, Ning Cheng, Jun Yu, Jing Xiao, Jianzong Wang

Comments: Accepted by 2024 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP2024)

Subjects: Computer Vision and Pattern Recognition (cs.CV); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[185] arXiv:2401.08095 (cross-list from cs.SD) [pdf, html, other]

Title: DurFlex-EVC: Duration-Flexible Emotional Voice Conversion Leveraging Discrete Representations without Text Alignment

Hyung-Seok Oh, Sang-Hoon Lee, Deok-Hyeon Cho, Seong-Whan Lee

Comments: 15 pages, 11 figures, 12 tables

Journal-ref: IEEE Transactions on Affective Computing, 2025, pp.1 - 15

Subjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Audio and Speech Processing (eess.AS)

[186] arXiv:2401.08096 (cross-list from cs.SD) [pdf, html, other]

Title: Learning Disentangled Speech Representations with Contrastive Learning and Time-Invariant Retrieval

Yimin Deng, Huaizhen Tang, Xulong Zhang, Ning Cheng, Jing Xiao, Jianzong Wang

Comments: Accepted by 2024 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP2024)

Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)

[187] arXiv:2401.08102 (cross-list from cs.SD) [pdf, other]

Title: DIFFRENT: A Diffusion Model for Recording Environment Transfer of Speech

Jaekwon Im, Juhan Nam

Comments: 4 pages, 2 figures

Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)

[188] arXiv:2401.08181 (cross-list from cs.SD) [pdf, other]

Title: LiveScaler: Live control of the harmony of an electronic music track

Alice Rixte (LaBRI, UB)

Comments: in French language

Journal-ref: Journ{\'e}es d'Informatique Musicale (JIM), Association Francophone d'Informatique Musicale (AFIM), May 2023, Saint-Denis (93), France

Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)

[189] arXiv:2401.08415 (cross-list from cs.SD) [pdf, other]

Title: From Coarse to Fine: Efficient Training for Audio Spectrogram Transformers

Jiu Feng, Mehmet Hamza Erol, Joon Son Chung, Arda Senocak

Comments: ICASSP 2024

Subjects: Sound (cs.SD); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

[190] arXiv:2401.08717 (cross-list from cs.SD) [pdf, html, other]

Title: Robust DOA estimation using deep acoustic imaging

Adrian S. Roman, Iran R. Roman, Juan P. Bello

Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)

[191] arXiv:2401.08835 (cross-list from cs.CL) [pdf, html, other]

Title: Improving ASR Contextual Biasing with Guided Attention

Jiyang Tang, Kwangyoun Kim, Suwon Shon, Felix Wu, Prashant Sridhar, Shinji Watanabe

Comments: Accepted at ICASSP 2024

Subjects: Computation and Language (cs.CL); Audio and Speech Processing (eess.AS)

[192] arXiv:2401.08851 (cross-list from cs.LG) [pdf, other]

Title: Using i-vectors for subject-independent cross-session EEG transfer learning

Jonathan Lasko, Jeff Ma, Mike Nicoletti, Jonathan Sussman-Fort, Sooyoung Jeong, William Hartmann

Comments: 11 pages

Subjects: Machine Learning (cs.LG); Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS); Neurons and Cognition (q-bio.NC)

[193] arXiv:2401.08887 (cross-list from cs.SD) [pdf, html, other]

Title: NOTSOFAR-1 Challenge: New Datasets, Baseline, and Tasks for Distant Meeting Transcription

Alon Vinnikov, Amir Ivry, Aviv Hurvitz, Igor Abramovski, Sharon Koubi, Ilya Gurvich, Shai Pe`er, Xiong Xiao, Benjamin Martinez Elizalde, Naoyuki Kanda, Xiaofei Wang, Shalev Shaer, Stav Yagev, Yossi Asher, Sunit Sivasankaran, Yifan Gong, Min Tang, Huaming Wang, Eyal Krupka

Comments: preprint

Subjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Audio and Speech Processing (eess.AS)

[194] arXiv:2401.08889 (cross-list from cs.SD) [pdf, html, other]

Title: On the Effect of Data-Augmentation on Local Embedding Properties in the Contrastive Learning of Music Audio Representations

Matthew C. McCallum, Matthew E. P. Davies, Florian Henkel, Jaehun Kim, Samuel E. Sandberg

Comments: Accepted to the International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2024

Subjects: Sound (cs.SD); Information Retrieval (cs.IR); Machine Learning (cs.LG); Multimedia (cs.MM); Audio and Speech Processing (eess.AS)

[195] arXiv:2401.08891 (cross-list from cs.SD) [pdf, html, other]

Title: Tempo estimation as fully self-supervised binary classification

Florian Henkel, Jaehun Kim, Matthew C. McCallum, Samuel E. Sandberg, Matthew E. P. Davies

Comments: Accepted to the International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2024

Subjects: Sound (cs.SD); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

[196] arXiv:2401.08902 (cross-list from cs.SD) [pdf, html, other]

Title: Similar but Faster: Manipulation of Tempo in Music Audio Embeddings for Tempo Prediction and Search

Matthew C. McCallum, Florian Henkel, Jaehun Kim, Samuel E. Sandberg, Matthew E. P. Davies

Comments: Accepted to the International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2024

Subjects: Sound (cs.SD); Digital Libraries (cs.DL); Information Retrieval (cs.IR); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

[197] arXiv:2401.08962 (cross-list from cs.HC) [pdf, html, other]

Title: DOO-RE: A dataset of ambient sensors in a meeting room for activity recognition

Hyunju Kim, Geon Kim, Taehoon Lee, Kisoo Kim, Dongman Lee

Subjects: Human-Computer Interaction (cs.HC); Machine Learning (cs.LG); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[198] arXiv:2401.08992 (cross-list from cs.CL) [pdf, html, other]

Title: Efficient Adapter Finetuning for Tail Languages in Streaming Multilingual ASR

Junwen Bai, Bo Li, Qiujia Li, Tara N. Sainath, Trevor Strohman

Comments: Accepted to ICASSP 2024

Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[199] arXiv:2401.09200 (cross-list from cs.SD) [pdf, html, other]

Title: A Real-Time Lyrics Alignment System Using Chroma And Phonetic Features For Classical Vocal Performance

Jiyun Park, Sangeon Yong, Taegyun Kwon, Juhan Nam

Comments: To Appear IEEE ICASSP 2024

Subjects: Sound (cs.SD); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

[200] arXiv:2401.09294 (cross-list from cs.SD) [pdf, html, other]

Title: T-FOLEY: A Controllable Waveform-Domain Diffusion Model for Temporal-Event-Guided Foley Sound Synthesis

Yoonjin Chung, Junwon Lee, Juhan Nam

Subjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS); Signal Processing (eess.SP)