Audio and Speech Processing

Authors and titles for January 2024

[201] arXiv:2401.09441 (cross-list from cs.SD) [pdf, html, other]

Title: Voxceleb-ESP: preliminary experiments detecting Spanish celebrities from their voices

Beltrán Labrador, Manuel Otero-Gonzalez, Alicia Lozano-Diez, Daniel Ramos, Doroteo T. Toledano, Joaquin Gonzalez-Rodriguez

Subjects: Sound (cs.SD); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

[202] arXiv:2401.09512 (cross-list from cs.SD) [pdf, html, other]

Title: MLAAD: The Multi-Language Audio Anti-Spoofing Dataset

Nicolas M. Müller, Piotr Kawa, Wei Herng Choong, Edresson Casanova, Eren Gölge, Thorsten Müller, Piotr Syga, Philip Sperl, Konstantin Böttinger

Comments: IJCNN 2024

Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)

[203] arXiv:2401.09752 (cross-list from cs.SD) [pdf, html, other]

Title: Improving Speaker-independent Speech Emotion Recognition Using Dynamic Joint Distribution Adaptation

Cheng Lu, Yuan Zong, Hailun Lian, Yan Zhao, Björn Schuller, Wenming Zheng

Comments: Accepted by ICASSP 2024

Subjects: Sound (cs.SD); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

[204] arXiv:2401.09759 (cross-list from cs.CV) [pdf, html, other]

Title: SlideAVSR: A Dataset of Paper Explanation Videos for Audio-Visual Speech Recognition

Hao Wang, Shuhei Kurita, Shuichiro Shimizu, Daisuke Kawahara

Comments: 3rd Workshop on Advances in Language and Vision Research (ALVR 2024)

Subjects: Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[205] arXiv:2401.09774 (cross-list from cs.MM) [pdf, html, other]

Title: On the Audio Hallucinations in Large Audio-Video Language Models

Taichi Nishimura, Shota Nakada, Masayoshi Kondo

Comments: 6 pages

Subjects: Multimedia (cs.MM); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[206] arXiv:2401.09880 (cross-list from cs.SD) [pdf, html, other]

Title: Attention-Based Recurrent Neural Network For Automatic Behavior Laying Hen Recognition

Fréjus A. A. Laleye, Mikaël A. Mousse

Subjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

[207] arXiv:2401.09988 (cross-list from cs.LG) [pdf, html, other]

Title: Developing an AI-based Integrated System for Bee Health Evaluation

Andrew Liang

Subjects: Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[208] arXiv:2401.10015 (cross-list from cs.CL) [pdf, html, other]

Title: Towards Hierarchical Spoken Language Dysfluency Modeling

Jiachen Lian, Gopala Anumanchipalli

Comments: 2024 EACL. Hierarchical extension of our previous workshop paper arXiv:2312.12810

Subjects: Computation and Language (cs.CL); Audio and Speech Processing (eess.AS)

[209] arXiv:2401.10070 (cross-list from cs.CL) [pdf, html, other]

Title: Communication-Efficient Personalized Federated Learning for Speech-to-Text Tasks

Yichao Du, Zhirui Zhang, Linan Yue, Xu Huang, Yuqing Zhang, Tong Xu, Linli Xu, Enhong Chen

Comments: ICASSP 2024

Subjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[210] arXiv:2401.10242 (cross-list from cs.OH) [pdf, html, other]

Title: DanceMeld: Unraveling Dance Phrases with Hierarchical Latent Codes for Music-to-Dance Synthesis

Xin Gao, Li Hu, Peng Zhang, Bang Zhang, Liefeng Bo

Comments: 10 pages, 8 figures

Subjects: Other Computer Science (cs.OH); Graphics (cs.GR); Human-Computer Interaction (cs.HC); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[211] arXiv:2401.10291 (cross-list from eess.SP) [pdf, html, other]

Title: Detecting Post-Stroke Aphasia Via Brain Responses to Speech in a Deep Learning Framework

Pieter De Clercq, Corentin Puffay, Jill Kries, Hugo Van Hamme, Maaike Vandermosten, Tom Francart, Jonas Vanthornhout

Comments: Shared first authors: De Clercq & Puffay

Subjects: Signal Processing (eess.SP); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[212] arXiv:2401.10446 (cross-list from cs.CL) [pdf, html, other]

Title: Large Language Models are Efficient Learners of Noise-Robust Speech Recognition

Yuchen Hu, Chen Chen, Chao-Han Huck Yang, Ruizhe Li, Chao Zhang, Pin-Yu Chen, EnSiong Chng

Comments: Accepted to ICLR 2024, Spotlight top 5%, 24 pages. This work will be open sourced at: this https URL under MIT license

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[213] arXiv:2401.10447 (cross-list from cs.CL) [pdf, html, other]

Title: Investigating Training Strategies and Model Robustness of Low-Rank Adaptation for Language Modeling in Speech Recognition

Yu Yu, Chao-Han Huck Yang, Tuan Dinh, Sungho Ryu, Jari Kolehmainen, Roger Ren, Denis Filimonov, Prashanth G. Shivakumar, Ankur Gandhe, Ariya Rastow, Jia Xu, Ivan Bulyko, Andreas Stolcke

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[214] arXiv:2401.10460 (cross-list from cs.SD) [pdf, html, other]

Title: Ultra-lightweight Neural Differential DSP Vocoder For High Quality Speech Synthesis

Prabhav Agrawal, Thilo Koehler, Zhiping Xiu, Prashant Serai, Qing He

Comments: Accepted for ICASSP 2024

Subjects: Sound (cs.SD); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

[215] arXiv:2401.10465 (cross-list from cs.CL) [pdf, html, other]

Title: Data-driven grapheme-to-phoneme representations for a lexicon-free text-to-speech

Abhinav Garg, Jiyeon Kim, Sushil Khyalia, Chanwoo Kim, Dhananjaya Gowda

Comments: Accepted at ICASSP 2024

Subjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[216] arXiv:2401.10544 (cross-list from cs.SD) [pdf, html, other]

Title: AAT: Adapting Audio Transformer for Various Acoustics Recognition Tasks

Yun Liang, Hai Lin, Shaojian Qiu, Yihang Zhang

Comments: Preprint version for ICASSP 2024, Korea

Subjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Audio and Speech Processing (eess.AS)

[217] arXiv:2401.10653 (cross-list from cs.CL) [pdf, html, other]

Title: Attentive Fusion: A Transformer-based Approach to Multimodal Hate Speech Detection

Atanu Mandal, Gargi Roy, Amit Barman, Indranil Dutta, Sudip Kumar Naskar

Comments: Accepted in 20th International Conference on Natural Language Processing (ICON)

Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG); Sound (cs.SD); Audio and Speech Processing (eess.AS); Signal Processing (eess.SP)

[218] arXiv:2401.10747 (cross-list from cs.SD) [pdf, html, other]

Title: Multimodal Sentiment Analysis with Missing Modality: A Knowledge-Transfer Approach

Weide Liu, Huijing Zhan, Hao Chen, Fengmao Lv

Comments: We request to withdraw our paper from the archive due to significant errors identified in the analysis and conclusions. Upon further review, we realized that these errors undermine the validity of our findings. We plan to conduct additional research to correct these issues and resubmit a revised version in the future

Subjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

[219] arXiv:2401.11095 (cross-list from cs.HC) [pdf, html, other]

Title: SoundShift: Exploring Sound Manipulations for Accessible Mixed-Reality Awareness

Ruei-Che Chang, Chia-Sheng Hung, Bing-Yu Chen, Dhruv Jain, Anhong Guo

Comments: DIS 2024

Subjects: Human-Computer Interaction (cs.HC); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[220] arXiv:2401.11102 (cross-list from cs.SD) [pdf, html, other]

Title: ASM: Audio Spectrogram Mixer

Qingfeng Ji, Jicun Zhang, Yuxin Wang

Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)

[221] arXiv:2401.11143 (cross-list from cs.LG) [pdf, html, other]

Title: Density Adaptive Attention is All You Need: Robust Parameter-Efficient Fine-Tuning Across Multiple Modalities

Georgios Ioannides, Aman Chadha, Aaron Elkins

Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV); Sound (cs.SD); Audio and Speech Processing (eess.AS); Signal Processing (eess.SP)

[222] arXiv:2401.11156 (cross-list from cs.CR) [pdf, other]

Title: Generalizing Speaker Verification for Spoof Awareness in the Embedding Space

Xuechen Liu, Md Sahidullah, Kong Aik Lee, Tomi Kinnunen

Comments: Published in IEEE/ACM Transactions on Audio, Speech, and Language Processing (doi updated)

Subjects: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[223] arXiv:2401.11199 (cross-list from cs.LG) [pdf, html, other]

Title: Projected Belief Networks With Discriminative Alignment for Acoustic Event Classification: Rivaling State of the Art CNNs

Paul M. Baggenstoss, Kevin Wilkinghoff, Felix Govaers, Frank Kurth

Comments: 15 Pages. Submitted to IEEE-TNNLS

Subjects: Machine Learning (cs.LG); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[224] arXiv:2401.11268 (cross-list from cs.CL) [pdf, html, other]

Title: Word-Level ASR Quality Estimation for Efficient Corpus Sampling and Post-Editing through Analyzing Attentions of a Reference-Free Metric

Golara Javadi, Kamer Ali Yuksel, Yunsu Kim, Thiago Castro Ferreira, Mohamed Al-Badrashiny

Journal-ref: 2024 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2024), Seoul, Korea

Subjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[225] arXiv:2401.11700 (cross-list from cs.CL) [pdf, html, other]

Title: Keep Decoding Parallel with Effective Knowledge Distillation from Language Models to End-to-end Speech Recognisers

Michael Hentschel, Yuta Nishikawa, Tatsuya Komatsu, Yusuke Fujita

Comments: Accepted at ICASSP 2024

Subjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[226] arXiv:2401.11983 (cross-list from cs.SD) [pdf, other]

Title: Lightweight Protection for Privacy in Offloaded Speech Understanding

Dongqi Cai

Comments: arXiv admin comment: This version has been removed by arXiv administrators as the submitter did not have the rights to agree to the license at the time of submission

Subjects: Sound (cs.SD); Cryptography and Security (cs.CR); Audio and Speech Processing (eess.AS)

[227] arXiv:2401.12039 (cross-list from cs.CV) [pdf, other]

Title: Look, Listen and Recognise: Character-Aware Audio-Visual Subtitling

Bruno Korbar, Jaesung Huh, Andrew Zisserman

Comments: Accepted for publication in ICASSP 2024

Subjects: Computer Vision and Pattern Recognition (cs.CV); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[228] arXiv:2401.12055 (cross-list from cs.CR) [pdf, other]

Title: NEUROSEC: FPGA-Based Neuromorphic Audio Security

Murat Isik, Hiruna Vishwamith, Yusuf Sur, Kayode Inadagbo, I. Can Dikmen

Comments: Audio processing, FPGA, Hardware Security, Neuromorphic Computing

Subjects: Cryptography and Security (cs.CR); Emerging Technologies (cs.ET); Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[229] arXiv:2401.12068 (cross-list from cs.SD) [pdf, other]

Title: Resource-constrained stereo singing voice cancellation

Clara Borrelli, James Rae, Dogac Basaran, Matt McVicar, Mehrez Souden, Matthias Mauch

Subjects: Sound (cs.SD); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

[230] arXiv:2401.12179 (cross-list from cs.SD) [pdf, html, other]

Title: DITTO: Diffusion Inference-Time T-Optimization for Music Generation

Zachary Novack, Julian McAuley, Taylor Berg-Kirkpatrick, Nicholas J. Bryan

Comments: Oral at ICML 2024

Subjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

[231] arXiv:2401.12266 (cross-list from cs.SD) [pdf, other]

Title: An Exploratory Study of Multimodal Physiological Data in Jazz Improvisation Using Basic Machine Learning Techniques

Yawen Zhang

Comments: Master's thesis

Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)

[232] arXiv:2401.12600 (cross-list from cs.SD) [pdf, html, other]

Title: EEND-M2F: Masked-attention mask transformers for speaker diarization

Marc Härkönen, Samuel J. Broughton, Lahiru Samarakoon

Comments: 14 pages, 2 figures

Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)

[233] arXiv:2401.12656 (cross-list from cs.SD) [pdf, html, other]

Title: MoodLoopGP: Generating Emotion-Conditioned Loop Tablature Music with Multi-Granular Features

Wenqian Cui, Pedro Sarmento, Mathieu Barthet

Comments: This preprint is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0). The Version of Record of this contribution is published in Proceedings of EvoMUSART: International Conference on Computational Intelligence in Music, Sound, Art and Design (Part of EvoStar) 2024

Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)

[234] arXiv:2401.12789 (cross-list from cs.CL) [pdf, html, other]

Title: Multilingual and Fully Non-Autoregressive ASR with Large Language Model Fusion: A Comprehensive Study

W. Ronny Huang, Cyril Allauzen, Tongzhou Chen, Kilol Gupta, Ke Hu, James Qin, Yu Zhang, Yongqiang Wang, Shuo-Yiin Chang, Tara N. Sainath

Comments: ICASSP 2024

Subjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[235] arXiv:2401.12925 (cross-list from cs.SD) [pdf, html, other]

Title: Emotion-Aware Contrastive Adaptation Network for Source-Free Cross-Corpus Speech Emotion Recognition

Yan Zhao, Jincen Wang, Cheng Lu, Sunan Li, Björn Schuller, Yuan Zong, Wenming Zheng

Comments: Accepted by ICASSP 2024

Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)

[236] arXiv:2401.12987 (cross-list from cs.CL) [pdf, html, other]

Title: TelME: Teacher-leading Multimodal Fusion Network for Emotion Recognition in Conversation

Taeyang Yun, Hyunkuk Lim, Jeonghwan Lee, Min Song

Comments: NAACL 2024 main conference

Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[237] arXiv:2401.12992 (cross-list from cs.CL) [pdf, html, other]

Title: TranSentence: Speech-to-speech Translation via Language-agnostic Sentence-level Speech Encoding without Language-parallel Data

Seung-Bin Kim, Sang-Hoon Lee, Seong-Whan Lee

Comments: Accepted by ICASSP 2024

Subjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[238] arXiv:2401.13260 (cross-list from cs.CL) [pdf, html, other]

Title: MF-AED-AEC: Speech Emotion Recognition by Leveraging Multimodal Fusion, Asr Error Detection, and Asr Error Correction

Jiajun He, Xiaohan Shi, Xingfeng Li, Tomoki Toda

Comments: Accepted by ICASSP 2024

Subjects: Computation and Language (cs.CL); Multimedia (cs.MM); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[239] arXiv:2401.13463 (cross-list from cs.CL) [pdf, html, other]

Title: SpeechDPR: End-to-End Spoken Passage Retrieval for Open-Domain Spoken Question Answering

Chyi-Jiunn Lin, Guan-Ting Lin, Yung-Sung Chuang, Wei-Lun Wu, Shang-Wen Li, Abdelrahman Mohamed, Hung-yi Lee, Lin-shan Lee

Comments: Accepted at ICASSP 2024

Subjects: Computation and Language (cs.CL); Information Retrieval (cs.IR); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[240] arXiv:2401.13498 (cross-list from cs.SD) [pdf, html, other]

Title: Expressive Acoustic Guitar Sound Synthesis with an Instrument-Specific Input Representation and Diffusion Outpainting

Hounsu Kim, Soonbeom Choi, Juhan Nam

Comments: Accepted to ICASSP 2024

Subjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS); Signal Processing (eess.SP)

[241] arXiv:2401.13527 (cross-list from cs.CL) [pdf, html, other]

Title: SpeechGPT-Gen: Scaling Chain-of-Information Speech Generation

Dong Zhang, Xin Zhang, Jun Zhan, Shimin Li, Yaqian Zhou, Xipeng Qiu

Comments: work in progress

Subjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[242] arXiv:2401.13548 (cross-list from cs.SD) [pdf, other]

Title: A Phoneme-Scale Assessment of Multichannel Speech Enhancement Algorithms

Nasser-Eddine Monir, Paul Magron, Romain Serizel

Comments: This is the preprint of the paper that we submitted to the Trends in Hearing Journal

Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)

[243] arXiv:2401.13611 (cross-list from cs.SD) [pdf, html, other]

Title: Non-Intrusive Speech Intelligibility Prediction for Hearing-Impaired Users using Intermediate ASR Features and Human Memory Models

Rhiannon Mogridge, George Close, Robert Sutherland, Thomas Hain, Jon Barker, Stefan Goetze, Anton Ragni

Comments: Accepted paper. IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP), Seoul, Korea, April 2024

Subjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Audio and Speech Processing (eess.AS)

[244] arXiv:2401.13851 (cross-list from cs.SD) [pdf, other]

Title: Scaling NVIDIA's Multi-speaker Multi-lingual TTS Systems with Zero-Shot TTS to Indic Languages

Akshit Arora, Rohan Badlani, Sungwon Kim, Rafael Valle, Bryan Catanzaro

Comments: Presentation accepted at ICASSP 2024

Subjects: Sound (cs.SD); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

[245] arXiv:2401.14185 (cross-list from cs.SD) [pdf, other]

Title: TDFNet: An Efficient Audio-Visual Speech Separation Model with Top-down Fusion

Samuel Pegg, Kai Li, Xiaolin Hu

Journal-ref: 2023 13th International Conference on Information Science and Technology (ICIST), Cairo, Egypt, 2023, pp. 243-252

Subjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Audio and Speech Processing (eess.AS)

[246] arXiv:2401.14289 (cross-list from cs.SD) [pdf, html, other]

Title: Speech foundation models on intelligibility prediction for hearing-impaired listeners

Santiago Cuervo, Ricard Marxer

Comments: To be presented in ICASSP 2024

Subjects: Sound (cs.SD); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

[247] arXiv:2401.14444 (cross-list from cs.SD) [pdf, html, other]

Title: ICASSP 2024 Speech Signal Improvement Challenge

Nicolae Catalin Ristea, Ando Saabas, Ross Cutler, Babak Naderi, Sebastian Braun, Solomiya Branets

Subjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Audio and Speech Processing (eess.AS)

[248] arXiv:2401.14542 (cross-list from cs.SD) [pdf, html, other]

Title: Exploring Musical Roots: Applying Audio Embeddings to Empower Influence Attribution for a Generative Music Model

Julia Barnett, Hugo Flores Garcia, Bryan Pardo

Comments: 14 pages + references. Under conference review

Subjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Audio and Speech Processing (eess.AS)

[249] arXiv:2401.14664 (cross-list from cs.SD) [pdf, html, other]

Title: UNIT-DSR: Dysarthric Speech Reconstruction System Using Speech Unit Normalization

Yuejiao Wang, Xixin Wu, Disong Wang, Lingwei Meng, Helen Meng

Comments: Accepted to ICASSP 2024

Subjects: Sound (cs.SD); Computation and Language (cs.CL); Audio and Speech Processing (eess.AS)

[250] arXiv:2401.14717 (cross-list from cs.CL) [pdf, html, other]

Title: Turn-taking and Backchannel Prediction with Acoustic and Large Language Model Fusion

Jinhan Wang, Long Chen, Aparna Khare, Anirudh Raju, Pranav Dheram, Di He, Minhua Wu, Andreas Stolcke, Venkatesh Ravichandran

Comments: To appear in IEEE ICASSP 2024

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Sound (cs.SD); Audio and Speech Processing (eess.AS)