-
Recreating Neural Activity During Speech Production with Language and Speech Model Embeddings
Authors:
Owais Mujtaba Khanday,
Pablo Rodroguez San Esteban,
Zubair Ahmad Lone,
Marc Ouellet,
Jose Andres Gonzalez Lopez
Abstract:
Understanding how neural activity encodes speech and language production is a fundamental challenge in neuroscience and artificial intelligence. This study investigates whether embeddings from large-scale, self-supervised language and speech models can effectively reconstruct high-gamma neural activity characteristics, key indicators of cortical processing, recorded during speech production. We le…
▽ More
Understanding how neural activity encodes speech and language production is a fundamental challenge in neuroscience and artificial intelligence. This study investigates whether embeddings from large-scale, self-supervised language and speech models can effectively reconstruct high-gamma neural activity characteristics, key indicators of cortical processing, recorded during speech production. We leverage pre-trained embeddings from deep learning models trained on linguistic and acoustic data to represent high-level speech features and map them onto these high-gamma signals. We analyze the extent to which these embeddings preserve the spatio-temporal dynamics of brain activity. Reconstructed neural signals are evaluated against high-gamma ground-truth activity using correlation metrics and signal reconstruction quality assessments. The results indicate that high-gamma activity can be effectively reconstructed using large language and speech model embeddings in all study participants, generating Pearson's correlation coefficients ranging from 0.79 to 0.99.
△ Less
Submitted 21 May, 2025; v1 submitted 20 May, 2025;
originally announced May 2025.
-
Simulation-driven engineering for the management of harmful algal and cyanobacterial blooms
Authors:
José L. Risco-Martín,
Segundo Esteban,
Jesús Chacón,
Gonzalo Carazo-Barbero,
Eva Besada-Portas,
José A. López-Orozco
Abstract:
Harmful Algal and Cyanobacterial Blooms (HABs), occurring in inland and maritime waters, pose threats to natural environments by producing toxins that affect human and animal health. In the past, HABs have been assessed mainly by the manual collection and subsequent analysis of water samples and occasionally by automatic instruments that acquire information from fixed locations. These procedures d…
▽ More
Harmful Algal and Cyanobacterial Blooms (HABs), occurring in inland and maritime waters, pose threats to natural environments by producing toxins that affect human and animal health. In the past, HABs have been assessed mainly by the manual collection and subsequent analysis of water samples and occasionally by automatic instruments that acquire information from fixed locations. These procedures do not provide data with the desirable spatial and temporal resolution to anticipate the formation of HABs. Hence, new tools and technologies are needed to efficiently detect, characterize and respond to HABs that threaten water quality. It is essential nowadays when the world's water supply is under tremendous pressure because of climate change, overexploitation, and pollution. This paper introduces DEVS-BLOOM, a novel framework for real-time monitoring and management of HABs. Its purpose is to support high-performance hazard detection with Model Based Systems Engineering (MBSE) and Cyber-Physical Systems (CPS) infrastructure for dynamic environments.
△ Less
Submitted 8 September, 2023;
originally announced September 2023.
-
IATos: AI-powered pre-screening tool for COVID-19 from cough audio samples
Authors:
D. Trejo Pizzo,
S. Esteban
Abstract:
OBJECTIVE: Our objective is to evaluate the possibility of using cough audio recordings (spontaneous or simulated) to detect sound patterns in people who are diagnosed with COVID-19. The research question that led our work was: what is the sensitivity and specificity of a machine learning based COVID-19 cough classifier, using RT-PCR tests as gold standard?
SETTING: The audio samples that were c…
▽ More
OBJECTIVE: Our objective is to evaluate the possibility of using cough audio recordings (spontaneous or simulated) to detect sound patterns in people who are diagnosed with COVID-19. The research question that led our work was: what is the sensitivity and specificity of a machine learning based COVID-19 cough classifier, using RT-PCR tests as gold standard?
SETTING: The audio samples that were collected for this study belong to individuals who were swabbed in the City of Buenos Aires in 20 public and 1 private facilities where RT-PCR studies were carried out on patients suspected of COVID, and 14 out-of-hospital isolation units for patients with confirmed COVID mild cases. The audios were collected through the Buenos Aires city government WhatsApp chatbot that was specifically designed to address citizen inquiries related to the coronavirus pandemic (COVID-19).
PARTICIPANTS: The data collected corresponds to 2821 individuals who were swabbed in the City of Buenos Aires, between August 11 and December 2, 2020. Individuals were divided into 1409 that tested positive for COVID-19 and 1412 that tested negative. From this sample group, 52.6% of the individuals were female and 47.4% were male. 2.5% were between the age of 0 and 20 , 61.1% between the age of 21 and 40 , 30.3% between the age of 41 and 60 and 6.1% were over 61 years of age.
RESULTS: Using the dataset of 2821 individuals our results showed that the neural network classifier was able to discriminate between the COVID-19 positive and the healthy coughs with an accuracy of 86%. This accuracy obtained during the training process was later tested and confirmed with a second dataset corresponding to 492 individuals.
△ Less
Submitted 9 December, 2021; v1 submitted 27 April, 2021;
originally announced April 2021.
