Audio and Speech Processing

Authors and titles for May 2023

[301] arXiv:2305.12868 (cross-list from cs.SD) [pdf, other]

Title: NAS-FM: Neural Architecture Search for Tunable and Interpretable Sound Synthesis based on Frequency Modulation

Zhen Ye, Wei Xue, Xu Tan, Qifeng Liu, Yike Guo

Journal-ref: IJCAI 2023

Subjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG); Multimedia (cs.MM); Audio and Speech Processing (eess.AS)

[302] arXiv:2305.12927 (cross-list from cs.CL) [pdf, other]

Title: Exploring Speaker-Related Information in Spoken Language Understanding for Better Speaker Diarization

Luyao Cheng, Siqi Zheng, Zhang Qinglin, Hui Wang, Yafeng Chen, Qian Chen

Comments: Accepted to Findings of ACL 2023

Subjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[303] arXiv:2305.13009 (cross-list from cs.CL) [pdf, other]

Title: Textually Pretrained Speech Language Models

Michael Hassid, Tal Remez, Tu Anh Nguyen, Itai Gat, Alexis Conneau, Felix Kreuk, Jade Copet, Alexandre Defossez, Gabriel Synnaeve, Emmanuel Dupoux, Roy Schwartz, Yossi Adi

Comments: NeurIPS 2023

Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[304] arXiv:2305.13033 (cross-list from cs.SD) [pdf, html, other]

Title: Towards generalizing deep-audio fake detection networks

Konstantin Gasenzer (1), Moritz Wolter (1) ((1) High Performance Computing and Analytics Lab, Universität Bonn, Germany)

Comments: Code available at: this https URL

Journal-ref: Published in Transactions on Machine Learning Research (04/2024)

Subjects: Sound (cs.SD); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

[305] arXiv:2305.13050 (cross-list from cs.SD) [pdf, other]

Title: AudioToken: Adaptation of Text-Conditioned Diffusion Models for Audio-to-Image Generation

Guy Yariv, Itai Gat, Lior Wolf, Yossi Adi, Idan Schwartz

Comments: Accepted to INTERSPEECH 2023

Subjects: Sound (cs.SD); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

[306] arXiv:2305.13080 (cross-list from cs.CL) [pdf, other]

Title: Mitigating Catastrophic Forgetting for Few-Shot Spoken Word Classification Through Meta-Learning

Ruan van der Merwe, Herman Kamper

Comments: 5 pages, 3 figures, Accepted to Interspeech 2023

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Audio and Speech Processing (eess.AS)

[307] arXiv:2305.13195 (cross-list from cs.SD) [pdf, other]

Title: U-DiT TTS: U-Diffusion Vision Transformer for Text-to-Speech

Xin Jing, Yi Chang, Zijiang Yang, Jiangjian Xie, Andreas Triantafyllopoulos, Bjoern W. Schuller

Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)

[308] arXiv:2305.13204 (cross-list from cs.CL) [pdf, other]

Title: Improving Isochronous Machine Translation with Target Factors and Auxiliary Counters

Proyag Pal, Brian Thompson, Yogesh Virkar, Prashant Mathur, Alexandra Chronopoulou, Marcello Federico

Comments: Accepted at INTERSPEECH 2023

Subjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[309] arXiv:2305.13210 (cross-list from cs.SD) [pdf, other]

Title: Learning to detect an animal sound from five examples

Inês Nolasco, Shubhr Singh, Veronica Morfi, Vincent Lostanlen, Ariana Strandburg-Peshkin, Ester Vidaña-Vila, Lisa Gill, Hanna Pamuła, Helen Whitehead, Ivan Kiskin, Frants H. Jensen, Joe Morford, Michael G. Emmerson, Elisabetta Versace, Emily Grout, Haohe Liu, Dan Stowell

Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS); Quantitative Methods (q-bio.QM)

[310] arXiv:2305.13213 (cross-list from cs.SD) [pdf, other]

Title: Computational models of sound-quality metrics using method for calculating loudness with gammatone/gammachirp auditory filterbank

Takuto Isoyama, Shunsuke Kidani, Masashi Unoki

Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)

[311] arXiv:2305.13262 (cross-list from cs.SD) [pdf, other]

Title: Modulation Extraction for LFO-driven Audio Effects

Christopher Mitcheltree, Christian J. Steinmetz, Marco Comunità, Joshua D. Reiss

Comments: Accepted to DAFx 2023. Listening samples and plugins can be found at this https URL

Subjects: Sound (cs.SD); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

[312] arXiv:2305.13512 (cross-list from cs.CL) [pdf, other]

Title: Can ChatGPT Detect Intent? Evaluating Large Language Models for Spoken Language Understanding

Mutian He, Philip N. Garner

Comments: 6 pages, 2 figures; Accepted by Interspeech 2023

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[313] arXiv:2305.13516 (cross-list from cs.CL) [pdf, other]

Title: Scaling Speech Technology to 1,000+ Languages

Vineel Pratap, Andros Tjandra, Bowen Shi, Paden Tomasello, Arun Babu, Sayani Kundu, Ali Elkahky, Zhaoheng Ni, Apoorv Vyas, Maryam Fazel-Zarandi, Alexei Baevski, Yossi Adi, Xiaohui Zhang, Wei-Ning Hsu, Alexis Conneau, Michael Auli

Subjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[314] arXiv:2305.13583 (cross-list from cs.CL) [pdf, other]

Title: Cross-Attention is Not Enough: Incongruity-Aware Dynamic Hierarchical Fusion for Multimodal Affect Recognition

Yaoting Wang, Yuanchao Li, Paul Pu Liang, Louis-Philippe Morency, Peter Bell, Catherine Lai

Comments: *First two authors contributed equally

Subjects: Computation and Language (cs.CL); Multimedia (cs.MM); Audio and Speech Processing (eess.AS); Image and Video Processing (eess.IV)

[315] arXiv:2305.13596 (cross-list from eess.IV) [pdf, other]

Title: Multimodal sensor fusion for real-time location-dependent defect detection in laser-directed energy deposition

Lequn Chen, Xiling Yao, Wenhe Feng, Youxiang Chew, Seung Ki Moon

Comments: 8 pages, 10 figures. This paper has been accepted to be published in the proceedings of IDETC-CIE 2023

Subjects: Image and Video Processing (eess.IV); Audio and Speech Processing (eess.AS); Signal Processing (eess.SP)

[316] arXiv:2305.13612 (cross-list from cs.SD) [pdf, other]

Title: FluentSpeech: Stutter-Oriented Automatic Speech Editing with Context-Aware Diffusion Models

Ziyue Jiang, Qian Yang, Jialong Zuo, Zhenhui Ye, Rongjie Huang, Yi Ren, Zhou Zhao

Comments: Accepted by ACL 2023 (Findings)

Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)

[317] arXiv:2305.13652 (cross-list from cs.CL) [pdf, other]

Title: Cross-lingual Knowledge Transfer and Iterative Pseudo-labeling for Low-Resource Speech Recognition with Transducers

Jan Silovsky, Liuhui Deng, Arturo Argueta, Tresi Arvizo, Roger Hsiao, Sasha Kuznietsov, Yiu-Chang Lin, Xiaoqiang Xiao, Yuanyuan Zhang

Subjects: Computation and Language (cs.CL); Audio and Speech Processing (eess.AS)

[318] arXiv:2305.13700 (cross-list from cs.SD) [pdf, other]

Title: Detection of Cross-Dataset Fake Audio Based on Prosodic and Pronunciation Features

Chenglong Wang, Jiangyan Yi, Jianhua Tao, Chuyuan Zhang, Shuai Zhang, Xun Chen

Comments: Interspeech2023

Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)

[319] arXiv:2305.13701 (cross-list from cs.SD) [pdf, other]

Title: TO-Rawnet: Improving RawNet with TCN and Orthogonal Regularization for Fake Audio Detection

Chenglong Wang, Jiangyan Yi, Jianhua Tao, Chuyuan Zhang, Shuai Zhang, Ruibo Fu, Xun Chen

Comments: Interspeech2023

Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)

[320] arXiv:2305.13713 (cross-list from cs.SD) [pdf, other]

Title: CALLS: Japanese Empathetic Dialogue Speech Corpus of Complaint Handling and Attentive Listening in Customer Center

Yuki Saito, Eiji Iimori, Shinnosuke Takamichi, Kentaro Tachibana, Hiroshi Saruwatari

Comments: 5 pages, accepted for INTERSPEECH2023

Subjects: Sound (cs.SD); Computation and Language (cs.CL); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

[321] arXiv:2305.13716 (cross-list from cs.SD) [pdf, other]

Title: BA-SOT: Boundary-Aware Serialized Output Training for Multi-Talker ASR

Yuhao Liang, Fan Yu, Yangze Li, Pengcheng Guo, Shiliang Zhang, Qian Chen, Lei Xie

Comments: Accepted by INTERSPEECH 2023

Subjects: Sound (cs.SD); Computation and Language (cs.CL); Audio and Speech Processing (eess.AS)

[322] arXiv:2305.13724 (cross-list from cs.SD) [pdf, other]

Title: ChatGPT-EDSS: Empathetic Dialogue Speech Synthesis Trained from ChatGPT-derived Context Word Embeddings

Yuki Saito, Shinnosuke Takamichi, Eiji Iimori, Kentaro Tachibana, Hiroshi Saruwatari

Comments: 5 pages, accepted for INTERSPEECH 2023

Subjects: Sound (cs.SD); Computation and Language (cs.CL); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

[323] arXiv:2305.13758 (cross-list from cs.SD) [pdf, other]

Title: A study of audio mixing methods for piano transcription in violin-piano ensembles

Hyemi Kim, Jiyun Park, Taegyun Kwon, Dasaem Jeong, Juhan Nam

Comments: To Appear IEEE ICASSP 2023

Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)

[324] arXiv:2305.13774 (cross-list from cs.SD) [pdf, other]

Title: ADD 2023: the Second Audio Deepfake Detection Challenge

Jiangyan Yi, Jianhua Tao, Ruibo Fu, Xinrui Yan, Chenglong Wang, Tao Wang, Chu Yuan Zhang, Xiaohui Zhang, Yan Zhao, Yong Ren, Le Xu, Junzuo Zhou, Hao Gu, Zhengqi Wen, Shan Liang, Zheng Lian, Shuai Nie, Haizhou Li

Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)

[325] arXiv:2305.13794 (cross-list from cs.CL) [pdf, other]

Title: Personalized Predictive ASR for Latency Reduction in Voice Assistants

Andreas Schwarz, Di He, Maarten Van Segbroeck, Mohammed Hethnawi, Ariya Rastrow

Comments: Accepted for Interspeech 2023

Subjects: Computation and Language (cs.CL); Audio and Speech Processing (eess.AS)

[326] arXiv:2305.13831 (cross-list from cs.SD) [pdf, other]

Title: ZET-Speech: Zero-shot adaptive Emotion-controllable Text-to-Speech Synthesis with Diffusion and Style-based Models

Minki Kang, Wooseok Han, Sung Ju Hwang, Eunho Yang

Comments: Accepted by INTERSPEECH 2023

Subjects: Sound (cs.SD); Computation and Language (cs.CL); Audio and Speech Processing (eess.AS)

[327] arXiv:2305.14023 (cross-list from cs.SD) [pdf, other]

Title: Happy or Evil Laughter? Analysing a Database of Natural Audio Samples

Aljoscha Düsterhöft, Felix Burkhardt, Björn W. Schuller

Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)

[328] arXiv:2305.14035 (cross-list from cs.LG) [pdf, other]

Title: Can Self-Supervised Neural Representations Pre-Trained on Human Speech distinguish Animal Callers?

Eklavya Sarkar, Mathew Magimai.-Doss

Comments: Accepted at Interspeech 2023

Subjects: Machine Learning (cs.LG); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[329] arXiv:2305.14042 (cross-list from cs.CL) [pdf, other]

Title: Improving speech translation by fusing speech and text

Wenbiao Yin, Zhicheng Liu, Chengqi Zhao, Tao Wang, Jian Tong, Rong Ye

Subjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[330] arXiv:2305.14049 (cross-list from cs.CL) [pdf, other]

Title: Rethinking Speech Recognition with A Multimodal Perspective via Acoustic and Semantic Cooperative Decoding

Tian-Hao Zhang, Hai-Bo Qin, Zhi-Hao Lai, Song-Lu Chen, Qi Liu, Feng Chen, Xinyuan Qian, Xu-Cheng Yin

Comments: Accepted by Interspeech 2023

Subjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[331] arXiv:2305.14071 (cross-list from cs.CL) [pdf, other]

Title: Disentangled Variational Autoencoder for Emotion Recognition in Conversations

Kailai Yang, Tianlin Zhang, Sophia Ananiadou

Comments: Accepted by IEEE Transactions on Affective Computing

Subjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[332] arXiv:2305.14097 (cross-list from cs.CR) [pdf, other]

Title: QFA2SR: Query-Free Adversarial Transfer Attacks to Speaker Recognition Systems

Guangke Chen, Yedi Zhang, Zhe Zhao, Fu Song

Comments: Accepted by the 32nd USENIX Security Symposium (2023 USENIX Security); Full Version

Subjects: Cryptography and Security (cs.CR); Machine Learning (cs.LG); Multimedia (cs.MM); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[333] arXiv:2305.14359 (cross-list from cs.MM) [pdf, other]

Title: Zero-shot personalized lip-to-speech synthesis with face image based voice control

Zheng-Yan Sheng, Yang Ai, Zhen-Hua Ling

Comments: ICASSP 2023

Subjects: Multimedia (cs.MM); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[334] arXiv:2305.14381 (cross-list from cs.LG) [pdf, other]

Title: Connecting Multi-modal Contrastive Representations

Zehan Wang, Yang Zhao, Xize Cheng, Haifeng Huang, Jiageng Liu, Li Tang, Linjun Li, Yongqi Wang, Aoxiong Yin, Ziang Zhang, Zhou Zhao

Comments: NeurIPS 2023

Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[335] arXiv:2305.14402 (cross-list from cs.SD) [pdf, html, other]

Title: Enhancing Speech Emotion Recognition Through Differentiable Architecture Search

Thejan Rajapakshe, Rajib Rana, Sara Khalifa, Berrak Sisman, Björn Schuller

Comments: 5 pages, 4 figures

Subjects: Sound (cs.SD); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

[336] arXiv:2305.14635 (cross-list from cs.CL) [pdf, other]

Title: CMOT: Cross-modal Mixup via Optimal Transport for Speech Translation

Yan Zhou, Qingkai Fang, Yang Feng

Comments: ACL 2023 main conference

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[337] arXiv:2305.14838 (cross-list from cs.CL) [pdf, other]

Title: ComSL: A Composite Speech-Language Model for End-to-End Speech-to-Text Translation

Chenyang Le, Yao Qian, Long Zhou, Shujie Liu, Yanmin Qian, Michael Zeng, Xuedong Huang

Comments: NeurIPS 2023, Poster

Subjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[338] arXiv:2305.14867 (cross-list from cs.SD) [pdf, other]

Title: Interactive Neural Resonators

Rodrigo Diaz, Charalampos Saitis, Mark Sandler

Subjects: Sound (cs.SD); Human-Computer Interaction (cs.HC); Audio and Speech Processing (eess.AS)

[339] arXiv:2305.14875 (cross-list from cs.HC) [pdf, other]

Title: LoopBoxes -- Evaluation of a Collaborative Accessible Digital Musical Instrument

Andreas Förster, Alarith Uhde, Mathias Komesker, Christina Komesker, Irina Schmidt

Comments: 10 pages, 9 figures, to be published in the Proceedings of the International Conference on New Interfaces for Musical Expression (NIME'23)

Subjects: Human-Computer Interaction (cs.HC); Computers and Society (cs.CY); Multimedia (cs.MM); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[340] arXiv:2305.15055 (cross-list from cs.SD) [pdf, other]

Title: Iteratively Improving Speech Recognition and Voice Conversion

Mayank Kumar Singh, Naoya Takahashi, Onoe Naoyuki

Subjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Audio and Speech Processing (eess.AS)

[341] arXiv:2305.15127 (cross-list from cs.SD) [pdf, other]

Title: PLCMOS -- a data-driven non-intrusive metric for the evaluation of packet loss concealment algorithms

Lorenz Diener, Marju Purin, Sten Sootla, Ando Saabas, Robert Aichner, Ross Cutler

Comments: to appear: INTERSPEECH 2023, associated model release: this https URL

Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)

[342] arXiv:2305.15255 (cross-list from cs.CL) [pdf, html, other]

Title: Spoken Question Answering and Speech Continuation Using Spectrogram-Powered LLM

Eliya Nachmani, Alon Levkovitch, Roy Hirsch, Julian Salazar, Chulayuth Asawaroengchai, Soroosh Mariooryad, Ehud Rivlin, RJ Skerry-Ryan, Michelle Tadmor Ramanovich

Comments: ICLR 2024 camera-ready

Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[343] arXiv:2305.15386 (cross-list from cs.CL) [pdf, other]

Title: Vistaar: Diverse Benchmarks and Training Sets for Indian Language ASR

Kaushal Santosh Bhogale, Sai Sundaresan, Abhigyan Raman, Tahir Javed, Mitesh M. Khapra, Pratyush Kumar

Comments: Accepted in INTERSPEECH 2023

Subjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[344] arXiv:2305.15403 (cross-list from cs.CL) [pdf, other]

Title: AV-TranSpeech: Audio-Visual Robust Speech-to-Speech Translation

Rongjie Huang, Huadai Liu, Xize Cheng, Yi Ren, Linjun Li, Zhenhui Ye, Jinzheng He, Lichao Zhang, Jinglin Liu, Xiang Yin, Zhou Zhao

Comments: Accepted to ACL 2023

Subjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[345] arXiv:2305.15405 (cross-list from cs.CL) [pdf, html, other]

Title: Textless Speech-to-Speech Translation With Limited Parallel Data

Anuj Diwan, Anirudh Srinivasan, David Harwath, Eunsol Choi

Comments: Accepted to EMNLP 2024 Findings

Subjects: Computation and Language (cs.CL); Audio and Speech Processing (eess.AS)

[346] arXiv:2305.15571 (cross-list from cs.SD) [pdf, other]

Title: Sound Design Strategies for Latent Audio Space Explorations Using Deep Learning Architectures

Kıvanç Tatar, Kelsey Cotton, Daniel Bisig

Comments: In Proceedings of Sound and Music Computing 2023, ISBN 978-91-527-7372-7

Subjects: Sound (cs.SD); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

[347] arXiv:2305.15601 (cross-list from cs.SD) [pdf, html, other]

Title: Metamathematics of Algorithmic Composition

Michael Gogins

Comments: 15 pages, 0 figures. Comments are very welcome

Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)

[348] arXiv:2305.15663 (cross-list from cs.CL) [pdf, other]

Title: Mixture-of-Expert Conformer for Streaming Multilingual ASR

Ke Hu, Bo Li, Tara N. Sainath, Yu Zhang, Francoise Beaufays

Comments: Accepted to Interspeech 2023

Subjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[349] arXiv:2305.15719 (cross-list from cs.SD) [pdf, other]

Title: Efficient Neural Music Generation

Max W. Y. Lam, Qiao Tian, Tang Li, Zongyu Yin, Siyuan Feng, Ming Tu, Yuliang Ji, Rui Xia, Mingbo Ma, Xuchen Song, Jitong Chen, Yuping Wang, Yuxuan Wang

Subjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

[350] arXiv:2305.15758 (cross-list from cs.SD) [pdf, other]

Title: Towards Solving Cocktail-Party: The First Method to Build a Realistic Dataset with Ground Truths for Speech Separation

Rawad Melhem, Assef Jafar, Oumayma Al Dakkak

Journal-ref: (2024) Romanian Journal of Acoustics and Vibration

Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)

[351] arXiv:2305.15760 (cross-list from cs.CL) [pdf, other]

Title: Svarah: Evaluating English ASR Systems on Indian Accents

Tahir Javed, Sakshi Joshi, Vignesh Nagarajan, Sai Sundaresan, Janki Nawale, Abhigyan Raman, Kaushal Bhogale, Pratyush Kumar, Mitesh M. Khapra

Subjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[352] arXiv:2305.15859 (cross-list from cs.SD) [pdf, other]

Title: Anomalous Sound Detection Based on Sound Separation

Kanta Shimonishi, Kota Dohi, Yohei Kawaguchi

Comments: Accepted to INTERSPEECH2023

Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)

[353] arXiv:2305.15898 (cross-list from cs.SD) [pdf, other]

Title: Room Impulse Response Estimation in a Multiple Source Environment

Kyungyun Lee, Jeonghun Seo, Keunwoo Choi, Sangmoon Lee, Ben Sangbae Chon

Comments: 2023 AES International Conference on Spatial and Immersive Audio

Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)

[354] arXiv:2305.15905 (cross-list from cs.SD) [pdf, other]

Title: Latent Diffusion Model Based Foley Sound Generation System For DCASE Challenge 2023 Task 7

Yi Yuan, Haohe Liu, Xubo Liu, Xiyuan Kang, Mark D. Plumbley, Wenwu Wang

Comments: DCASE 2023 task 7 technical report, ranked 1st in the challenge

Subjects: Sound (cs.SD); Multimedia (cs.MM); Audio and Speech Processing (eess.AS)

[355] arXiv:2305.15937 (cross-list from cs.CL) [pdf, other]

Title: Visually grounded few-shot word acquisition with fewer shots

Leanne Nortje, Benjamin van Niekerk, Herman Kamper

Comments: Accepted at Interspeech 2023

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Audio and Speech Processing (eess.AS)

[356] arXiv:2305.16043 (cross-list from cs.SD) [pdf, other]

Title: Ordered and Binary Speaker Embedding

Jiaying Wang, Xianglong Wang, Namin Wang, Lantian Li, Dong Wang

Comments: to be published in INTERSPEECH 2023

Subjects: Sound (cs.SD); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

[357] arXiv:2305.16049 (cross-list from cs.CV) [pdf, other]

Title: CN-Celeb-AV: A Multi-Genre Audio-Visual Dataset for Person Recognition

Lantian Li, Xiaolou Li, Haoyu Jiang, Chen Chen, Ruihai Hou, Dong Wang

Comments: INTERSPEECH 2023

Subjects: Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[358] arXiv:2305.16070 (cross-list from cs.SD) [pdf, other]

Title: Visualizing data augmentation in deep speaker recognition

Pengqi Li, Lantian Li, Askar Hamdulla, Dong Wang

Comments: to be published in INTERSPEECH 2023

Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)

[359] arXiv:2305.16093 (cross-list from cs.CL) [pdf, other]

Title: End-to-End Simultaneous Speech Translation with Differentiable Segmentation

Shaolei Zhang, Yang Feng

Comments: Accepted at ACL 2023 findings

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[360] arXiv:2305.16107 (cross-list from cs.CL) [pdf, other]

Title: VioLA: Unified Codec Language Models for Speech Recognition, Synthesis, and Translation

Tianrui Wang, Long Zhou, Ziqiang Zhang, Yu Wu, Shujie Liu, Yashesh Gaur, Zhuo Chen, Jinyu Li, Furu Wei

Comments: Working in progress

Subjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[361] arXiv:2305.16263 (cross-list from cs.SD) [pdf, other]

Title: Unified Modeling of Multi-Talker Overlapped Speech Recognition and Diarization with a Sidecar Separator

Lingwei Meng, Jiawen Kang, Mingyu Cui, Haibin Wu, Xixin Wu, Helen Meng

Comments: Accepted to INTERSPEECH 2023

Subjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

[362] arXiv:2305.16333 (cross-list from cs.CL) [pdf, other]

Title: Text Generation with Speech Synthesis for ASR Data Augmentation

Zhuangqun Huang, Gil Keren, Ziran Jiang, Shashank Jain, David Goss-Grubbs, Nelson Cheng, Farnaz Abtahi, Duc Le, David Zhang, Antony D'Avirro, Ethan Campbell-Taylor, Jessie Salas, Irina-Elena Veliche, Xi Chen

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

[363] arXiv:2305.16342 (cross-list from cs.CL) [pdf, other]

Title: InterFormer: Interactive Local and Global Features Fusion for Automatic Speech Recognition

Zhi-Hao Lai, Tian-Hao Zhang, Qi Liu, Xinyuan Qian, Li-Fang Wei, Song-Lu Chen, Feng Chen, Xu-Cheng Yin

Comments: Accepted by Interspeech 2023

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[364] arXiv:2305.16371 (cross-list from cs.CL) [pdf, other]

Title: INTapt: Information-Theoretic Adversarial Prompt Tuning for Enhanced Non-Native Speech Recognition

Eunseop Yoon, Hee Suk Yoon, John Harvill, Mark Hasegawa-Johnson, Chang D. Yoo

Comments: ACL2023

Subjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[365] arXiv:2305.16445 (cross-list from cs.SD) [pdf, other]

Title: SoundSieve: Seconds-Long Audio Event Recognition on Intermittently-Powered Systems

Mahathir Monjur, Yubo Luo, Zhenyu Wang, Shahriar Nirjon

Comments: The 21st ACM International Conference on Mobile Systems, Applications, and Services (Mobisys 2023)

Subjects: Sound (cs.SD); Distributed, Parallel, and Cluster Computing (cs.DC); Audio and Speech Processing (eess.AS)

[366] arXiv:2305.16592 (cross-list from cs.SD) [pdf, other]

Title: A Multi-Scale Attentive Transformer for Multi-Instrument Symbolic Music Generation

Xipin Wei, Junhui Chen, Zirui Zheng, Li Guo, Lantian Li, Dong Wang

Comments: to be published in INTERSPEECH 2023

Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)

[367] arXiv:2305.16664 (cross-list from cs.CL) [pdf, other]

Title: Score-balanced Loss for Multi-aspect Pronunciation Assessment

Heejin Do, Yunsu Kim, Gary Geunbae Lee

Comments: Accepted at Interspeech 2023

Subjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[368] arXiv:2305.16749 (cross-list from cs.SD) [pdf, other]

Title: Diverse and Expressive Speech Prosody Prediction with Denoising Diffusion Probabilistic Model

Xiang Li, Songxiang Liu, Max W. Y. Lam, Zhiyong Wu, Chao Weng, Helen Meng

Comments: Proceedings of Interspeech 2023 (doi: https://doi.org/10.21437/Interspeech.2023-715), demo site at this https URL

Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)

[369] arXiv:2305.16897 (cross-list from cs.CL) [pdf, other]

Title: Inter-connection: Effective Connection between Pre-trained Encoder and Decoder for Speech Translation

Yuta Nishikawa, Satoshi Nakamura

Comments: Accepted at INTERSPEECH2023

Subjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[370] arXiv:2305.16932 (cross-list from cs.SD) [pdf, other]

Title: A Neural State-Space Model Approach to Efficient Speech Separation

Chen Chen, Chao-Han Huck Yang, Kai Li, Yuchen Hu, Pin-Jui Ku, Eng Siong Chng

Comments: Accepted by InterSpeech 2023

Subjects: Sound (cs.SD); Computation and Language (cs.CL); Audio and Speech Processing (eess.AS)

[371] arXiv:2305.16957 (cross-list from cs.CL) [pdf, other]

Title: DisfluencyFixer: A tool to enhance Language Learning through Speech To Speech Disfluency Correction

Vineet Bhat, Preethi Jyothi, Pushpak Bhattacharyya

Comments: To be published in Interspeech 2023 - Show and Tell Demonstrations

Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[372] arXiv:2305.17000 (cross-list from cs.SD) [pdf, html, other]

Title: DistriBlock: Identifying adversarial audio samples by leveraging characteristics of the output distribution

Matías Pizarro, Dorothea Kolossa, Asja Fischer

Comments: Available at: this https URL

Journal-ref: Proceedings of the Fortieth Conference on Uncertainty in Artificial Intelligence, PMLR 244:2956-2988, 2024

Subjects: Sound (cs.SD); Cryptography and Security (cs.CR); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

[373] arXiv:2305.17191 (cross-list from cs.LG) [pdf, other]

Title: MT-SLVR: Multi-Task Self-Supervised Learning for Transformation In(Variant) Representations

Calum Heggan, Tim Hospedales, Sam Budgett, Mehrdad Yaghoobi

Comments: Last author version accepted to InterSpeech23. 5 pages

Subjects: Machine Learning (cs.LG); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[374] arXiv:2305.17343 (cross-list from cs.CV) [pdf, other]

Title: Modality-Independent Teachers Meet Weakly-Supervised Audio-Visual Event Parser

Yung-Hsuan Lai, Yen-Chun Chen, Yu-Chiang Frank Wang

Comments: NeurIPS 2023

Subjects: Computer Vision and Pattern Recognition (cs.CV); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[375] arXiv:2305.17499 (cross-list from cs.CL) [pdf, other]

Title: CIF-PT: Bridging Speech and Text Representations for Spoken Language Understanding via Continuous Integrate-and-Fire Pre-Training

Linhao Dong, Zhecheng An, Peihao Wu, Jun Zhang, Lu Lu, Zejun Ma

Comments: Accepted by ACL 2023 Findings

Subjects: Computation and Language (cs.CL); Multimedia (cs.MM); Audio and Speech Processing (eess.AS)

[376] arXiv:2305.17547 (cross-list from cs.CL) [pdf, other]

Title: Translatotron 3: Speech to Speech Translation with Monolingual Data

Eliya Nachmani, Alon Levkovitch, Yifan Ding, Chulayuth Asawaroengchai, Heiga Zen, Michelle Tadmor Ramanovich

Comments: To appear in ICASSP 2024

Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[377] arXiv:2305.17651 (cross-list from cs.CL) [pdf, other]

Title: DPHuBERT: Joint Distillation and Pruning of Self-Supervised Speech Models

Yifan Peng, Yui Sudo, Shakeel Muhammad, Shinji Watanabe

Comments: Accepted at INTERSPEECH 2023. Code will be available at: this https URL

Subjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[378] arXiv:2305.17706 (cross-list from cs.SD) [pdf, other]

Title: Spot keywords from very noisy and mixed speech

Ying Shi, Dong Wang, Lantian Li, Jiqing Han, Shi Yin

Comments: Interspeech 2023

Subjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Audio and Speech Processing (eess.AS)

[379] arXiv:2305.17732 (cross-list from cs.SD) [pdf, other]

Title: StyleS2ST: Zero-shot Style Transfer for Direct Speech-to-speech Translation

Kun Song, Yi Ren, Yi Lei, Chunfeng Wang, Kun Wei, Lei Xie, Xiang Yin, Zejun Ma

Comments: Accepted to Interspeech 2023

Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)

[380] arXiv:2305.17733 (cross-list from cs.CL) [pdf, other]

Title: Investigating Pre-trained Audio Encoders in the Low-Resource Condition

Hao Yang, Jinming Zhao, Gholamreza Haffari, Ehsan Shareghi

Comments: INTERSPEECH 2023

Subjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[381] arXiv:2305.17739 (cross-list from cs.SD) [pdf, other]

Title: Range-Based Equal Error Rate for Spoof Localization

Lin Zhang, Xin Wang, Erica Cooper, Nicholas Evans, Junichi Yamagishi

Comments: Accepted to Interspeech 2023

Subjects: Sound (cs.SD); Computation and Language (cs.CL); Audio and Speech Processing (eess.AS)

[382] arXiv:2305.17749 (cross-list from cs.SD) [pdf, other]

Title: Bayesian inference and neural estimation of acoustic wave propagation

Yongchao Huang, Yuhang He, Hong Ge

Subjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Audio and Speech Processing (eess.AS); Data Analysis, Statistics and Probability (physics.data-an)

[383] arXiv:2305.17758 (cross-list from cs.SD) [pdf, other]

Title: CAPTDURE: Captioned Sound Dataset of Single Sources

Yuki Okamoto, Kanta Shimonishi, Keisuke Imoto, Kota Dohi, Shota Horiguchi, Yohei Kawaguchi

Comments: Accepted to INTERSPEECH2023

Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)

[384] arXiv:2305.17782 (cross-list from cs.CL) [pdf, other]

Title: RASR2: The RWTH ASR Toolkit for Generic Sequence-to-sequence Speech Recognition

Wei Zhou, Eugen Beck, Simon Berger, Ralf Schlüter, Hermann Ney

Comments: accepted at Interspeech 2023

Subjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[385] arXiv:2305.17834 (cross-list from cs.SD) [pdf, html, other]

Title: Streaming Audio Transformers for Online Audio Tagging

Heinrich Dinkel, Zhiyong Yan, Yongqing Wang, Junbo Zhang, Yujun Wang, Bin Wang

Comments: Interspeech2024

Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)

[386] arXiv:2305.17846 (cross-list from cs.SD) [pdf, other]

Title: Retraining-free Customized ASR for Enharmonic Words Based on a Named-Entity-Aware Model and Phoneme Similarity Estimation

Yui Sudo, Kazuya Hata, Kazuhiro Nakadai

Comments: accepted by INTERSPEECH2023

Subjects: Sound (cs.SD); Computation and Language (cs.CL); Audio and Speech Processing (eess.AS)

[387] arXiv:2305.17860 (cross-list from cs.SD) [pdf, other]

Title: speech and noise dual-stream spectrogram refine network with speech distortion loss for robust speech recognition

Haoyu Lu, Nan Li, Tongtong Song, Longbiao Wang, Jianwu Dang, Xiaobao Wang, Shiliang Zhang

Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)

[388] arXiv:2305.17993 (cross-list from cs.SD) [pdf, other]

Title: Multi-Scale Attention for Audio Question Answering

Guangyao Li, Yixin Xu, Di Hu

Comments: Accepted by InterSpeech 2023

Subjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Multimedia (cs.MM); Audio and Speech Processing (eess.AS)

[389] arXiv:2305.18011 (cross-list from cs.CL) [pdf, other]

Title: Can We Trust Explainable AI Methods on ASR? An Evaluation on Phoneme Recognition

Xiaoliang Wu, Peter Bell, Ajitha Rajan

Subjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[390] arXiv:2305.18028 (cross-list from cs.SD) [pdf, other]

Title: ADAPTERMIX: Exploring the Efficacy of Mixture of Adapters for Low-Resource TTS Adaptation

Ambuj Mehrish, Abhinav Ramesh Kashyap, Li Yingting, Navonil Majumder, Soujanya Poria

Comments: Interspeech 2023

Subjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Audio and Speech Processing (eess.AS)

[391] arXiv:2305.18045 (cross-list from cs.SD) [pdf, other]

Title: Few-shot Class-incremental Audio Classification Using Adaptively-refined Prototypes

Wei Xie, Yanxiong Li, Qianhua He, Wenchang Cao, Tuomas Virtanen

Comments: 5 pages,2 figures, Accepted by Interspeech 2023

Subjects: Sound (cs.SD); Multimedia (cs.MM); Audio and Speech Processing (eess.AS)

[392] arXiv:2305.18096 (cross-list from cs.CL) [pdf, other]

Title: Improving Textless Spoken Language Understanding with Discrete Units as Intermediate Target

Guan-Wei Wu, Guan-Ting Lin, Shang-Wen Li, Hung-yi Lee

Comments: Accepted by Interspeech 2023. *Equal contribution

Subjects: Computation and Language (cs.CL); Audio and Speech Processing (eess.AS)

[393] arXiv:2305.18108 (cross-list from cs.SD) [pdf, other]

Title: Exploration of Efficient End-to-End ASR using Discretized Input from Self-Supervised Learning

Xuankai Chang, Brian Yan, Yuya Fujita, Takashi Maekaku, Shinji Watanabe

Comments: Accepted at INTERSPEECH 2023

Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)

[394] arXiv:2305.18281 (cross-list from cs.CL) [pdf, other]

Title: HyperConformer: Multi-head HyperMixer for Efficient Speech Recognition

Florian Mai, Juan Zuluaga-Gomez, Titouan Parcollet, Petr Motlicek

Comments: Florian Mai and Juan Zuluaga-Gomez contributed equally. To appear in Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH 2023

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

[395] arXiv:2305.18283 (cross-list from cs.CL) [pdf, other]

Title: CommonAccent: Exploring Large Acoustic Pretrained Models for Accent Classification Based on Common Voice

Juan Zuluaga-Gomez, Sara Ahmed, Danielius Visockas, Cem Subakan

Comments: To appear in Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH 2023

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

[396] arXiv:2305.18355 (cross-list from cs.SD) [pdf, other]

Title: An Efficient Membership Inference Attack for the Diffusion Model by Proximal Initialization

Fei Kong, Jinhao Duan, RuiPeng Ma, Hengtao Shen, Xiaofeng Zhu, Xiaoshuang Shi, Kaidi Xu

Subjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

[397] arXiv:2305.18392 (cross-list from cs.SD) [pdf, other]

Title: Speech Intelligibility Assessment of Dysarthric Speech by using Goodness of Pronunciation with Uncertainty Quantification

Eun Jung Yeo, Kwanghee Choi, Sunhee Kim, Minhwa Chung

Comments: Accepted to Interspeech 2023

Subjects: Sound (cs.SD); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

[398] arXiv:2305.18419 (cross-list from cs.CL) [pdf, other]

Title: Semantic Segmentation with Bidirectional Language Models Improves Long-form ASR

W. Ronny Huang, Hao Zhang, Shankar Kumar, Shuo-yiin Chang, Tara N. Sainath

Comments: Interspeech 2023. First 3 authors contributed equally

Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG); Sound (cs.SD); Audio and Speech Processing (eess.AS)

[399] arXiv:2305.18474 (cross-list from cs.SD) [pdf, other]

Title: Make-An-Audio 2: Temporal-Enhanced Text-to-Audio Generation

Jiawei Huang, Yi Ren, Rongjie Huang, Dongchao Yang, Zhenhui Ye, Chen Zhang, Jinglin Liu, Xiang Yin, Zejun Ma, Zhou Zhao

Subjects: Sound (cs.SD); Machine Learning (cs.LG); Multimedia (cs.MM); Audio and Speech Processing (eess.AS)

[400] arXiv:2305.18500 (cross-list from cs.CV) [pdf, other]

Title: VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset

Sihan Chen, Handong Li, Qunbo Wang, Zijia Zhao, Mingzhen Sun, Xinxin Zhu, Jing Liu

Comments: Accepted by NeurIPS 2023

Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)