-
Tackling Cognitive Impairment Detection from Speech: A submission to the PROCESS Challenge
Authors:
Catarina Botelho,
David Gimeno-Gómez,
Francisco Teixeira,
John Mendonça,
Patrícia Pereira,
Diogo A. P. Nunes,
Thomas Rolland,
Anna Pompili,
Rubén Solera-Ureña,
Maria Ponte,
David Martins de Matos,
Carlos-D. Martínez-Hinarejos,
Isabel Trancoso,
Alberto Abad
Abstract:
This work describes our group's submission to the PROCESS Challenge 2024, with the goal of assessing cognitive decline through spontaneous speech, using three guided clinical tasks. This joint effort followed a holistic approach, encompassing both knowledge-based acoustic and text-based feature sets, as well as LLM-based macrolinguistic descriptors, pause-based acoustic biomarkers, and multiple ne…
▽ More
This work describes our group's submission to the PROCESS Challenge 2024, with the goal of assessing cognitive decline through spontaneous speech, using three guided clinical tasks. This joint effort followed a holistic approach, encompassing both knowledge-based acoustic and text-based feature sets, as well as LLM-based macrolinguistic descriptors, pause-based acoustic biomarkers, and multiple neural representations (e.g., LongFormer, ECAPA-TDNN, and Trillson embeddings). Combining these feature sets with different classifiers resulted in a large pool of models, from which we selected those that provided the best balance between train, development, and individual class performance. Our results show that our best performing systems correspond to combinations of models that are complementary to each other, relying on acoustic and textual information from all three clinical tasks.
△ Less
Submitted 30 December, 2024;
originally announced January 2025.
-
Vizaj -- An interactive javascript tool for visualizing spatial networks
Authors:
Thibault Rolland,
Fabrizio De Vico Fallani
Abstract:
In many fields of science and technology we are confronted with complex networks. Making sense of these networks often require the ability to visualize and explore their intermingled structure consisting of nodes and links. To facilitate the identification of significant connectivity patterns, many methods have been developed based on the rearrangement of the nodes so as to avoid link criss-cross.…
▽ More
In many fields of science and technology we are confronted with complex networks. Making sense of these networks often require the ability to visualize and explore their intermingled structure consisting of nodes and links. To facilitate the identification of significant connectivity patterns, many methods have been developed based on the rearrangement of the nodes so as to avoid link criss-cross. However, real networks are often embedded in a geometrical space and the nodes code for an intrinsic physical feature of the system that one might want to preserve. For these spatial networks, it is therefore crucial to find alternative strategies operating on the links and not on the nodes. Here, we introduce Vizaj a javascript web application to visualize spatial networks based on optimized geometrical criteria that reshape the link profiles. While optimized for 3D networks, Vizaj can also be used for 2D networks and offers the possibility to interactively customize the visualization via several controlling parameters, including network filtering and the effect of internode distance on the link trajectories. Vizaj is further equipped with additional options allowing to improve the final aesthetics, such as the color/size of both nodes and links, zooming/rotating/translating, and superimposing external objects. Vizaj is an open-source software which can be freely downloaded and updated via a github repository. Here, we provide a detailed description of its main features and algorithms together with a guide on how to use it. Finally, we validate its potential on several synthetic and real spatial networks from infrastructural to biological systems. We hope that Vizaj will help scientists and practitioners to make sense of complex networks and provide aesthetic while informative visualizations.
△ Less
Submitted 27 October, 2022;
originally announced December 2022.
-
The INESC-ID Multi-Modal System for the ADReSS 2020 Challenge
Authors:
Anna Pompili,
Thomas Rolland,
Alberto Abad
Abstract:
This paper describes a multi-modal approach for the automatic detection of Alzheimer's disease proposed in the context of the INESC-ID Human Language Technology Laboratory participation in the ADReSS 2020 challenge. Our classification framework takes advantage of both acoustic and textual feature embeddings, which are extracted independently and later combined. Speech signals are encoded into acou…
▽ More
This paper describes a multi-modal approach for the automatic detection of Alzheimer's disease proposed in the context of the INESC-ID Human Language Technology Laboratory participation in the ADReSS 2020 challenge. Our classification framework takes advantage of both acoustic and textual feature embeddings, which are extracted independently and later combined. Speech signals are encoded into acoustic features using DNN speaker embeddings extracted from pre-trained models. For textual input, contextual embedding vectors are first extracted using an English Bert model and then used either to directly compute sentence embeddings or to feed a bidirectional LSTM-RNNs with attention. Finally, an SVM classifier with linear kernel is used for the individual evaluation of the three systems. Our best system, based on the combination of linguistic and acoustic information, attained a classification accuracy of 81.25%. Results have shown the importance of linguistic features in the classification of Alzheimer's Disease, which outperforms the acoustic ones in terms of accuracy. Early stage features fusion did not provide additional improvements, confirming that the discriminant ability conveyed by speech in this case is smooth out by linguistic data.
△ Less
Submitted 29 May, 2020;
originally announced May 2020.
-
Pathological speech detection using x-vector embeddings
Authors:
Catarina Botelho,
Francisco Teixeira,
Thomas Rolland,
Alberto Abad,
Isabel Trancoso
Abstract:
The potential of speech as a non-invasive biomarker to assess a speaker's health has been repeatedly supported by the results of multiple works, for both physical and psychological conditions. Traditional systems for speech-based disease classification have focused on carefully designed knowledge-based features. However, these features may not represent the disease's full symptomatology, and may e…
▽ More
The potential of speech as a non-invasive biomarker to assess a speaker's health has been repeatedly supported by the results of multiple works, for both physical and psychological conditions. Traditional systems for speech-based disease classification have focused on carefully designed knowledge-based features. However, these features may not represent the disease's full symptomatology, and may even overlook its more subtle manifestations. This has prompted researchers to move in the direction of general speaker representations that inherently model symptoms, such as Gaussian Supervectors, i-vectors and, x-vectors. In this work, we focus on the latter, to assess their applicability as a general feature extraction method to the detection of Parkinson's disease (PD) and obstructive sleep apnea (OSA). We test our approach against knowledge-based features and i-vectors, and report results for two European Portuguese corpora, for OSA and PD, as well as for an additional Spanish corpus for PD. Both x-vector and i-vector models were trained with an out-of-domain European Portuguese corpus. Our results show that x-vectors are able to perform better than knowledge-based features in same-language corpora. Moreover, while x-vectors performed similarly to i-vectors in matched conditions, they significantly outperform them when domain-mismatch occurs.
△ Less
Submitted 31 May, 2020; v1 submitted 2 March, 2020;
originally announced March 2020.
